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ABSTRACT 


Autonomous  vehicles  with  plume  tracing  capabilities  would  be  valuable  for  finding 
chemical  sources  in  fluid  flows.  This  article  considers  strategies  allowing  autonomous 
vehicles  to  find  and  trace  an  odor  plume  to  its  source.  These  strategies  are  inspired  by  the 
maneuvers  of  moths  flying  upwind  along  a  pheromone  plume.  Although  moth  maneuvers 
are  well  documented,  the  mechanisms  underlying  sensory  perception  and  navigation  are 
not  fully  understood;  therefore,  a  key  objective  was  to  define  sensor,  signal  processing,  and 
actuation  algorithms  for  autonomous  vehicles.  The  strategies  presented  do  not  precisely 
mimic  insect  orientation  to  odors.  Optimizing  performance,  however,  suggests  orientation 
strategies  that  may  have  biological  counterparts.  The  results  demonstrate  the  importance  of 
cross-plume  countertuming  strategies  for  maintaining  intermittent  contact  with  the 
chemical  plume,  given  noisy  sensory  information.  It  is  important  for  the  searcher  to 
maintain  intermittent  contact  with  the  plume  because  flow  direction  while  detecting  odor  is 
the  main  indicator  of  the  instantaneous  desired  direction  of  motion. 
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1  INTRODUCTION 


Olfactory-based  mechanisms  have  been  hypothesized  for  a  variety  of  biological  behaviors 
(Dusenbery,  1992;  Vickers,  2000;  Zimmer  and  Butman,  2000):  homing  by  Pacific  salmon  (Hasler 
and  Scholz,  1983),  foraging  by  Antarctic  procellariiform  seabirds  (Nevitt,  2000),  foraging  by 
lobsters  (Atema,  1995;  Basil  and  Atema,  1994;  Devine  and  Atema,  1982),  foraging  by  blue  crabs 
(Weissburg  and  Zimmer-Faust,  1994),  and  mate-seeking  and  foraging  by  insects  (Carde,  1996). 
Typically,  olfactory-based  mechanisms  proposed  for  biological  entities  combine  a  large-scale 
orientation  behavior  based  in  part  on  olfaction  with  a  multisensor  local  search  in  the  vicinity  of 
the  source.  The  long-range  olfactory  based  search  is  documented  in  moths  at  ranges  of  100-1000 
m  (Rau  and  Rau,  1929;  Elkinton,  et  al.,  1987)  and  in  Antarctic  procellariiform  seabirds  over 
thousands  of  kilometers  (Nevitt,  2000). 

Recently  there  has  been  interest  in  the  development  of  algorithms  to  replicate  these  feats  on 
autonomous  vehicles  (e.g.,  Larcombe,  1984;  Kuwana,  et  al.,  1999;  Ishida,  et  al.,  2001).  The  goal 
of  the  autonomous  vehicle  would  be  to  locate  the  source  of  a  chemical  that  is  transported  in  a 
turbulent  fluid  flow.  Such  autonomous  vehicle  capabilities  have  applicability  in  searching  for 
environmentally  interesting  phenomena,  unexploded  ordinance,  undersea  wreckage,  and  sources 
of  hazardous  chemicals  or  pollutants.  In  this  article,  we  develop  strategies,  inspired  by  moth 
plume  following  behaviors,  that  allow  an  autonomous  vehicle  to  trace  a  chemical  plume  from  a 
possible  large  initial  distance  to  the  odor  source. 

The  location  of  pheromone-emitting  females  by  flying  male  moths  is  considered  a 
remarkable  case  of  odor-guided  navigation.  Upwind  flight  along  pheromone  plumes  is  guided  by 
two  complementary  mechanisms.  First,  direction  with  respect  to  wind  flow  is  gauged  by  the 
optomotor  reaction:  feedback  from  movement  in  the  visual  surround  provides  information  about 


the  wind’s  direction  and  the  organism’s  progress  upwind.  This  maneuver  is  termed  odor-induced 
optomotor  anemotaxis.  Second,  the  precise  upwind  heading  and  velocity  seem  to  be  governed 
mainly  by  moment-to-moment  contact  with  small-scale  filaments  of  pheromone.  Filament 
encounter  rates  above  5  Hz  promote  rapid  flight  with  a  mainly  due-upwind  heading  (Mafra-Neto 
and  Carde,  1994;  Vickers  and  Baker,  1994a).  When  the  filament  encounter  rate  falls  below  5  Hz, 
males  tend  to  fly  a  slower  zigzag  course,  countertuming  across  the  wind  with  a  reduced  rate  of 
upwind  displacement. 

Male  moths  that  lose  contact  with  the  plume  ‘cast,’  that  is,  they  cease  upwind  movement 
and  their  crosswind  excursions  progressively  widen  (Kuenen  and  Carde,  1994).  Casting  can 
continue  for  several  seconds  until  contact  with  the  plume  is  re-established.  If  moths  fail  to  re¬ 
enter  the  plume,  they  may  return  to  ‘ranging  behaviors’  used  for  initial  location  of  a  plume 
(Elkinton  and  Carde,  1983).  There  are  two  principal  reasons  that  upwind  flight  within  the  plume 
can  lead  to  loss  of  contact  with  odorant  (Murlis,  Elkinton  and  Carde,  1992).  First,  the  internal 
structure  of  plumes  is  patchy,  that  is,  in  certain  locations  the  signal  is  not  present  or  it  is  below 
the  threshold  of  detection,  two  effects  that  are  magnified  as  the  plume  is  carried  downwind. 
Second,  the  upwind  direction  may  lead  out  of  the  plume  due  to  the  meander  of  the  plume 
centerline.  In  this  article,  the  three  types  of  behavior  described  above  are  referred  to  as  plume 
finding  (^ranging),  plume  maintaining  (wupwind  orientation)  and  plume  reacquisition  (^casting). 

Strategies  of  walking  and  flying  organisms  following  a  fluid-advected  plume  differ  in  one 
important  respect.  In  walking  organisms,  the  direction  of  fluid  flow  is  sensed  by 
mechanoreceptors  (e.g.,  comparative  deflection  of  the  two  antennae),  with  reference  to  the 
organism’s  position  on  the  walking  substrate  (Bell,  1984).  The  straightforward  mechanism  for 
detection  of  fluid  directionality  used  by  walking  insects  is  not  available  to  flying  insects,  but 


could  be  readily  employed  by  a  ground-based  autonomous  vehicle. 

Because  the  first  male  moth  to  locate  a  female  typically  secures  a  mating,  there  is 
considerable  selective  pressure  for  males  to  be  efficient  at  locating  a  plume  of  pheromone  and 
then  to  quickly  navigate  a  course  up  the  plume  to  the  female  (Carde,  1986).  Therefore,  a 
biological  system’s  plume-tracing  strategy  should  be  optimized,  given  the  sensor  and  mobility 
constraints  of  the  biological  system.  It  should  also  be  robust  to  typical  variations  in 
environmental  conditions.  The  presumed  optimality  and  robustness  of  biologically-based  plume 
tracing  strategies  are  valuable  traits  to  mimic  in  an  autonomous  vehicle  designed  to  locate  a 
source  of  an  odor  (e.g.,  a  source  of  environment  contamination)  via  plume  tracing. 

Much  of  what  we  know  about  plume  following  maneuvers  of  flying  insects  is  based  on 
studies  of  moths  flying  along  odor  plumes  generated  in  wind  tunnels.  In  the  natural  environment, 
where  the  fluid  flow  is  more  complex,  systematic  behavioral  studies  are  difficult  to  perform. 
Translation  of  the  observed  insect  behavior  into  an  algorithm  for  plume  tracing  suitable  for 
autonomous  vehicle  applications  is  still  in  its  infancy. 

Robotic  tracking  of  chemical  trails  is  discussed  in  e.g.  Deveza  et  al.  (1994).  Tracing  of 
chemical  plumes  is  studied  in  several  articles  (e.g.,  Belanger  and  Willis,  1996a,  1996b,  1998; 
Belanger  and  Arbas,  1998;  Grasso,  et  al.,  2000;  Ishida,  et  al.,  1996,  1998,  1999;  Kuwana,  et  al. 
1999;  Rozas,  1991;  Russell,  1995;  Russell,  2001).  Belanger  and  Willis  (1996a,  1996b,  1998)  and 
Belanger  and  Arbas  (1998)  follow  an  approach  similar  to  that  used  herein.  Belanger,  et  al. 
implement  proposed  strategies  as  software  algorithms  (see  the  appendix  of  (Belanger  and  Arbas, 
1998))  that  can  be  evaluated  in  computer-based  simulations.  The  main  advantages  of  this 
approach  are  that  alternative  strategies  can  be  evaluated  under  identical  experimental  conditions 
and  the  causes  of  the  searcher  maneuvers  (i.e.,  central  nervous  system  processing  of  living 


entities)  are  known.  The  main  disadvantage,  relative  to  experimental  evaluation,  is  that  the 
validity  of  the  results  is  a  function  of  the  extent  to  which  the  characteristics  of  the  environment 
are  accurately  modeled.  The  simulated  environment  of  Belanger  and  Willis  (1996a,  1996b,  1998) 
and  Belanger  and  Arbas  (1998)  is  similar  to  a  small  wind  tunnel.  The  centerline  of  the  plume 
aligns  with  the  mean  wind  vector  and  points  directly  back  to  the  source.  Three  strategies  are 
proposed  in  Belanger  and  Arbas  (1998).  These  will  be  further  discussed  in  Section  6. 

Grasso,  Consi,  Mountain  and  Atema  (2000)  present  and  analyze  chemotaxis 
(concentration  information  only)  based  algorithms  using  two  concentration  sensors  mounted  on  a 
robot  lobster.  The  article  includes  results  of  experiments  in  a  flume  where  their  robo-lobster 
initiated  its  search  1.0  m  or  less  from  the  source  in  the  downstream  center  of  a  well-developed 
plume.  Grasso,  et  al.  (2000)  discusses  some  of  the  experimental  difficulties  that  result  from  the 
intermittency  due  to  a  turbulent  flow.  Because  the  experiments  occur  in  a  flume  they  are  not  able 
to  analyze  the  challenges  imposed  by  plumes  with  significant  meander. 

Ishida  et  al.  (1996)  present  on-vehicle  experimental  results  which  use  an  array  of  sensors 
calculating  a  cross-plume  concentration  gradient  to  track  the  plume  centerline.  Ishida  et  al. 
(1998)  propose  a  method  to  remotely  estimate  the  source  location  by  using  a  Kalman  Filter  and 
nonlinear  optimization  methods  to  estimate  parameters  of  a  Gaussian  plume  distribution  model. 
It  is  interesting,  in  comparison  with  the  counter-turning  algorithms  suggested  herein,  that 
accurate  estimation  of  the  desired  parameters  by  this  method  requires  the  use  of  cross-plume  ‘Z’ 
shaped  paths.  Although  not  directly  comparable  to  the  results  herein,  the  algorithms  of  Ishida  et 
al.  (1998)  required  on  the  order  of  300  s  to  trace  a  2  m  plume.  This  slow  rate  of  travel  was 
required  due  to  slow  sensor  characteristics  and  the  need  to  calculated  statistical  properties  and 
gradient  information.  The  approaches  presented  herein  can  respond  more  quickly  because  their 


decisions  are  based  on  instantaneous  concentration  measurements.  The  remote  estimation 
approach  in  Ishida  et  al.  (1998)  is  similar  to  that  suggested  in  Stacey  (2000).  Ishida  (1999) 
presents  a  multi-sensor  methodology  for  estimating  the  3-D  direction  to  the  odor  source.  The 
method  assumes  that  the  local  time-averaged  concentration  gradient  at  the  vehicle  location  points 
toward  the  odor  source.  This  assumption  is  only  true  in  the  vicinity  (a  few  meters)  of  the  odor 
source,  at  greater  distances  slow  changes  in  the  direction  of  the  wind  will  cause  the  plume 
centerline  to  meander  (see  e.g.  Figure  2).  Also,  at  large  distances  from  the  source  detection  of  the 
plume  gradient  might  require  a  large  inter-sensor  separation. 

Kuwana  et  al.  (1999)  present  experimental  results  for  a  miniature  robot  instrumented  with 
silkworm  antennae  and  using  a  chemotaxis  strategy  tracing  a  10  cm  pheromone  plume  in  a  wind 
tunnel.  The  focus  of  this  article  was  the  use  of  antennae  as  pheromone  sensors.  Rozas  (1991) 
presents  an  electronic  nose  design,  sensor  processing  algorithms,  and  gradient  following 
strategies.  Russell  (1995)  includes  robotic  implementation  of  algorithms  that  estimate  statistics 
of  the  plume  such  as  the  plume  centroid.  The  strategy  then  maneuvers  relative  to  the  plume 
centroid  in  an  upwind  direction.  Such  strategies  are  not  expected  to  work  well  in  situations 
where  the  flow  can  change  significantly  resulting  in  plume  meander,  because  the  centroid  would 
then  be  time  varying.  Russell  (2001)  also  includes  robotic  implementation  experiments  where 
the  chemical  is  constrained  to  a  multiple  duct  tunnel  system.  The  approach  focuses  on  selecting 
the  correct  tunnels  to  follow  to  locate  the  odor  source. 

In  this  paper,  we  investigate  the  problem  of  maintaining  contact  with  a  plume  once  it  has 
been  detected.  In  contrast  to  the  articles  cited  above  that  studied  plume  tracing  near  the  source 
where  the  wind  direction  and  plume  axis  are  aligned,  we  focus  on  the  development  of  effective 
algorithms  for  tracking  the  plume  at  distances  from  the  source  where  it  is  possible  that  neither  the 


plume  centerline  nor  the  wind  direction  point  to  the  source.  The  article  is  organized  as  follows. 
Section  2  reviews  key  issues  from  the  topic  of  chemical  dispersion  in  turbulent  media  and 
summarizes  the  plume  model  used  in  the  Monte  Carlo  simulation  studies  of  this  article.  Section 
3  proposes  a  strategy  to  maintain  contact  with  the  plume,  presents  the  variables  that  will  be  used 
to  quantify  performance,  and  evaluates  how  changes  in  the  key  strategy  parameters  affect 
performance.  Section  4  proposes  and  optimizes  a  strategy  with  improved  ability  to  maintain 
contact  with  the  intermittent  plume.  The  optimized  strategies  that  result  from  Section  4  yield 
generally  zigzag  shaped  ground  tracks  similar  in  form  to  those  exhibited  by  a  range  of  biological 
entities  during  plume  tracking.  Section  5  evaluates  plume  tracking  performance  using  fluid  flow 
and  concentration  data  obtained  from  flume  experiments.  Section  6  compares  virtual  plume 
tracking  strategies  with  strategies  used  by  insects  and  other  organisms.  The  appendix  to  this 
article  includes  a  table  defining  various  notations  used  in  the  body  of  the  article. 

2  PLUMES  AND  PLUME-MAINTAINING  BEHAVIOR 
Moth  plume-tracing  behavior  can  be  used  as  a  basis  for  designing  an  autonomous  system  to 
search  for  the  unknown  location  of  an  odor  source;  however,  the  sensitivity  of  search 
performance  to  various  environmental  factors  and  strategy  parameters  must  be  investigated 
before  accepting  such  biologically-based  search  strategies  as  appropriate  for  autonomous 
systems.  For  example,  we  should  examine  how  we  can  alter  strategy  parameters  to  decrease  the 
time  to  find  the  plume,  increase  the  time  in  contact  with  the  plume,  and  reduce  the  time  to 
reacquire  the  plume  when  contact  is  lost.  This  article  focuses  on  the  problem  of  maximizing  the 
time  in  contact  with  the  plume  while  making  progress  toward  the  source  and  being  robust  to 
changes  in  environmental  conditions.  Monte  Carlo  simulation  methods  are  used  for  performance 
evaluation.  Both  simulated  air  plumes  and  experimental  water  flume  data  were  used  for  the 


performance  evaluations.  The  simulated  plumes  are  more  difficult  to  track  than  the  plumes  in  the 
flume  data  sets  due  to  the  significant  meander  of  the  plume  centerline  that  can  be  included  in  the 
simulations  by  temporal  and  spatial  variation  of  the  wind  direction  and  velocity.  Therefore,  the 
simulated  plumes  are  used  throughout  the  development  portions  of  this  study.  Important  aspects 
of  diffusion  in  a  turbulent  media  are  summarized  in  Subsection  2.1.  Subsection  2.2  summarizes 
important  aspects  of  the  plume  simulation  implemented  for  these  studies.  Experimental  flume 
data  are  used  for  the  performance  evaluations  presented  in  Section  5  to  ensure  that  the  strategy 
performance  has  not  inadvertently  been  optimized  to  the  characteristics  of  the  simulation. 

2.1  Diffusion  in  a  Turbulent  Media 

Pheromone  is  transported  downwind  by  advection.  The  pheromone  is  dispersed  mainly  by 
the  forces  of  turbulence  while  it  is  transported  downwind  in  turbulent  eddies,  which  range 
upward  in  scale  from  centimeters.  Individual  filaments  of  pheromone  may  be  transported 
relatively  undiluted  for  many  meters  before  being  tom  apart  and  mixed  with  clean  air  by  small- 
scale  turbulence  (Murlis  and  Jones,  1981;  Murlis  et  al.,  1992,  Murlis,  Willis  and  Carde,  2000). 

A  typical  time  series  of  concentration  at  a  fixed  downwind  location  is  shown  in  Fig.  1 .  The 
turbulent  nature  of  the  transport  mechanism  causes  the  sensed  concentration  to  be  only 
intermittently  above  a  positive  sensor  threshold  t.  Such  intermittency  has  several  consequences. 
First,  algorithms  based  on  instantaneous  gradient  following  in  an  intermittent  plume  would  be 
extremely  inefficient.  The  instantaneous  gradient  varies  too  rapidly  to  be  calculated.  The 
gradient  of  a  mean  concentration  theoretically  could  be  traced;  however,  plume  meander,  the 
required  spatial  sampling,  and  the  requirement  of  a  long  averaging  time  prohibit  on-vehicle 
calculation  of  the  mean  concentration  function  and,  therefore,  gradient-search  methods.  Second, 


intermittency  causes  the  concepts  of  ‘in  the  plume’  and  ‘not  in  the  plume’  to  be  difficult  to 
define,  as  discussed  in  the  following  paragraphs. 

Conventional  time-averaged  concentration  models  (e.g.,  Sutton,  1953)  are  time 
independent.  Such  models  assume  that  the  plume  was  fully  developed  prior  to  the  initiation  of 
data  collection.  Concentration  distributions  (assuming  stationarity  and  ergodicity)  of  such  models 
allow  time  independent  predictions,  over  an  ensemble  of  plumes,  of  the  average  concentration  as 
a  function  of  position.  However,  measured  instantaneous  concentrations  of  specific  exemplar 
plumes  deviate  significantly  from  the  ensemble  average  (Aylor,  1976;  Aylor,  Parlange,  and 
Granett,  1976;  Jones,  1983;  Mylne,  1992).  This  deviation  is  the  result  of  the  intermittency  and  of 
the  plume  meander.  Because  flying  moths  are  making  behavioral  decisions  using  odor 
concentrations  sensed  over  fractions  of  a  second,  whereas  time-averaged  models  calculate 
concentrations  over  a  span  of  several  minutes,  time-averaged  concentration  models  are  not  useful 
for  behavioral  studies  (Elkinton,  Carde  and  Mason,  1984).  Time-averaged  concentrations  also 
are  not  useful  as  a  basis  for  maneuver  design.  A  fleet  of  searchers  would  be  required  to 
implement  the  spatial  sampling  and  the  time  averaging  would  prohibit  rapid  plume  tracing.-  We 
instead  pursue  an  approach  where  an  individual  searcher  tracks  a  plume  by  reacting  to  the  sensed 
instantaneous  concentration. 

For  the  strategies  developed  herein,  a  single  search  vehicle  with  a  wind  direction  and  a 
binary  concentration  sensor  is  assumed.  The  sensed  concentration  at  time  t  is  denoted  c(t).  The 
searcher  will  consider  itself  to  be  instantaneously  ‘in  the  plume’  at  time  t  if  it  has  detected  above 
threshold  odor  within  the  last  Tw  seconds  (i.e.,  The  searcher  is  in  the  plume  at  time  t  if  c(t-T)>z, 

for  some  T  such  that  0  <T<  Tw.) '. 


The  studies  to  follow  use  a  plume  simulation  model  that  generates  individual  realizations 
of  odor  plumes2.  The  simulated  plume  concentration  can  be  evaluated  at  any  location  with  a  time 
resolution  of  0.01  seconds.  Such  simulated  plume  models  enable  certain  types  of  analysis  that 
would  be  precluded  by  the  loss  of  time  signature  information  in  mean  concentration  plume 
models  and  allows  performance  analysis  and  optimization  under  repeatable  conditions  that  are 
not  possible  with  plumes  in  natural  environments  [the  tradeoffs  between  simulation  and  flight 
testing  are  discussed  by  Belanger  and  Arbas  (1998)]. 

2.2  Plume  Simulation  Model 

To  generate  realistic  simulated  plumes  we  considered  the  following  three  characteristic 
structural  features  to  be  of  primary  concern.  First,  the  sensed  plume  at  a  fixed  location  should 
have  an  intermittent  internal  structure  that  closely  duplicates  experimental  fine  time-scale 
observations.  Second,  the  plume  shape  that  results  from  a  semi-continuous  release  of  odor 
should  be  sinuous  and  time  varying.  Third,  the  plume  shape  and  fluid  flow  history  should  be 
coherent. 

The  first  characteristic  feature  is  achieved  by  representing  the  odor  plume  as  a  sequence  of 
puffs.  Puffs  are  released  sequentially  at  the  source  location.  Each  puff  is  composed  of  n 
filaments.  After  release,  each  filament  center  location  is  determined  as  a  function  of  time  by 
integration  of  the  advective  fluid  flow  velocity.  To  achieve  the  second  characteristic  feature,  the 
simulation  incorporates  a  dynamic  model  of  the  advective  fluid  flow  that  as  the  solution  of  a  K- 
closure  based  partial  differential  equation  (Pielke,  1984)  driven  by  boundary  conditions  that  are 
implemented  as  colored  noise  processes  (p.  123  in  Jazwinski,  1970).  As  the  boundary  conditions 
change,  the  partial  differential  equation  is  solved  numerically  over  the  region  of  interest.  This 
model  allows  the  fluid  flow,  which  advects  the  filaments,  to  be  a  continuous  but  varying  function 
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of  position  and  time.  Changes  in  fluid  flow  direction  cause  the  plume  to  meander  (form  a  snake¬ 
like  path  when  viewed  from  above).  Because  the  fluid  flow  direction  and  magnitude  change, 
spatially  and  temporally,  the  instantaneous  fluid  flow  direction  within  the  plume  often  will  not 
point  toward  the  odor's  source  nor  be  coincident  with  the  plume's  centerline.  The  meander  and 
lack  of  correlation  between  the  wind  direction  and  the  plume  long  axis  in  the  simulated  plumes 
reproduce  similar  features  in  odor  plumes  observed  in  the  field  (Elkinton  et  al.,  1987;  Brady, 
Gibson  and  Packer,  1989)  that  significantly  complicate  the  plume  tracking  process. 

A  detailed  description  of  this  filament-based  odor  dispersion  model  is  contained  in  (Farrell, 
Murlis,  Long,  Li  and  Carde,  2002).  The  resulting  plumes  duplicate  both  the  time-averaged  and 
instantaneous  features  of  plume  structures  measured  in  the  field  (Jones,  1983)  without  sacrificing 
computational  feasibility. 

Related  plume  models  based  on  the  advection  of  puffs  have  been  suggested  in  previous 
work.  In  (Gifford,  1959),  puffs  are  represented  as  two  dimensional  (i.e.,  planar)  disks,  but  the 
advection  model  was  not  specified.  The  model  of  Belanger  and  Arbas  (1998)  represents  the 
puffs  as  homogeneous  patches  and  appears  to  assume  a  constant  unidirectional  fluid  flow  model. 
The  fluid  flow  model  used  in  this  study  represents  a  significant  extension  to  such  puff-based 
plume  models. 

The  filament-based  model,  presented  in  detail  in  (Farrell  et  al.,  2002),  is  general  and  is 
designed  so  that  it  can  be  tuned  for  a  range  of  applications.  For  the  simulation  results  presented 
here,  the  simulation  parameters  were  set  to  correspond  to  odor  plumes  in  wind  as  perceived  by 
male  moths.  The  receptor  system  of  some  male  moths  is  capable  of  resolving  10  odor  pulses  per 
second  (Rumbo  and  Kaissling,  1989).  A  flying  male  can  react  to  odor  filaments  within  200  ms 
(Mafra-Neto  and  Carde,  1994,  1996).  Therefore,  we  designed  the  present  model  of  plume 
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dispersion  with  an  integration  time  step  of  10  ms.  Similarly,  the  units  of  the  sensed  concentration 
were  specified  relative  to  the  measured  threshold  of  the  male  gypsy  moth  (Lymantria  dispar) 
(Elkinton  et  al.,  1984)  and  the  rate  of  emission  at  the  mean  rate  of  pheromone  release  by  the 
female  (Charlton  and  Carde,  1982). 

A  graphic  ‘snapshot’  illustrating  the  nature  of  the  plume  that  results  from  this  simulation 
model  is  contained  in  Fig.  2.  This  figure  shows  a  vehicle,  with  heading  v| /,  searching  for  the 
plume.  The  search  area  is  a  rectangle  with  comers:  ( x,y ),  (x,y) ,  (x,y),  and  (5c,  y).  For  Fig.  2,  x  = 

0,  y  =  -50,  x  =  100,  and  y—  50  m.  The  gray  scale  indicates  the  local  odor  concentration  as  a 


function  of  position.  Meander  of  the  plume  is  evident  clearly  as  is  dispersion  relative  to  the 
meandering  plume  centerline.  The  array  of  small  arrows  on  the  figure  indicates  the  direction  of 
the  local  fluid  flow  vector  at  the  tail  of  each  arrow.  Note  that  the  fluid  flow  vector  varies 
spatially  and  that  the  correlation  of  the  fluid  flow  direction  and  plume  axis  direction  decreases 
with  the  distance  from  the  source.  The  source  location  is  a  simulation  variable.  For  the 
simulation  studies  in  this  article,  the  source  will  be  located  at  (20,0)  (Fig.  2)  to  allow  analysis  of 
strategy  performance  when  the  searcher  starts  upwind  from  the  source. 
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threshold  was  0.01;  the  filament  release  rate  was  3  filaments  s'1;  the  simulation  time  step  was 
0.01  s;  and,  the  mean  fluid  velocity  was  1ms'1. 

2.3  Fluid  flow  Variation 

An  important  issue  in  Monte  Carlo  analysis  is  performance  sensitivity  to  variations  in  the 
parameters  of  the  fluid  flow  model,  particularly  those  that  affect  meander.  Note  that  the 
simulation  model  produces  a  fluid  flow  field  that  varies  continuously  as  a  function  of  time  and 
space.  The  model  parameters  determine  the  rate  and  magnitude  of  these  changes.  Therefore,  the 
analyses  to  follow  use  two  settings  of  the  fluid  flow  model  parameters  that  will  be  referred  to  as 
‘large  meander’  and  ‘small  meander.’  The  small  meander  parameter  settings  yield  plumes  with 
meander  shapes  typified  as  in  Fig.  2.  The  typical  plume  has  instantaneous  centerline  lateral 
deviation  from  the  mean  centerline  (i.e.,  the  x-axis)  at  y=100  that  are  normally  less  than  10  m. 
The  large  meander  parameter  settings  yield  plumes  with  instantaneous  centerline  lateral  deviation 
from  the  mean  centerline  at  7=100  that  are  normally  less  that  25  m.  Both  sets  of  parameter 
settings  generate  plumes  of  realistic  extent  in  the  y  direction.  The  plumes  that  result  with  the 
small  meander  parameter  settings  are  significantly  easier  to  track. 

3  ANALYSIS  OF  MAINTAINING  BEHAVIOR 

The  plume  tracing  task  is  decomposed  into  the  four  components:  finding  the  plume, 
maintaining  (intermittent)  contact  with  the  plume,  reacquiring  contact  with  the  plume,  and 
‘declaring’  that  the  source  has  been  found.  The  focus  of  this  article  is  on  the  strategies  for 
maintaining  and  reacquiring  contact  with  the  plume. 

From  an  initial  starting  position,  the  first  objective  is  to  find  the  plume.  The  plume  is  found 
once  the  first  above  threshold  odor  concentration  c(t)  is  detected.  The  total  time  spent  finding  the 
plume  is  denoted  by  7>.  The  searcher  has  no  prior  information  about  the  likely  odor  source 


location  within  the  search  area;  therefore,  plume  finding  task  is  an  uninformed  search  problem. 
There  are  many  known  solutions  for  uninformed  search  problems  (Dusenbery,  1992).  The  plume 
finding  strategy  used  throughout  this  article  is  described  in  the  appendix.  When  the  area  to  be 
searched  is  large,  finding  the  plume  can  consume  a  large  amount  of  resources  (i.e.,  time  and 
energy). 

Once  the  plume  has  been  found  the  objective  is  to  maintain  contact  with  it  while  making 
progress  toward  the  source.  The  total  time  in  the  plume,  referred  to  as  the  maintain  time,  will  be 
denoted  by  TM.  If  the  searcher  loses  contact  with  the  plume,  then  the  behavior  switches  to  one 
that  is  likely  to  reacquire  contact  with  the  plume  so  that  the  searcher  does  not  need  to  revert  to 
the  (resource-consuming)  plume-finding  behavior.  The  total  time  spent  trying  to  reacquire  the 
plume  will  be  denoted  by  TR.  Finally,  the  searcher  should  be  able  to  declare  that  the  source  has 
been  found.  In  biological  entities,  the  source  found  declaration  is  often  influence  by  multiple 
sensory  modalities,  including  visual  and  tactile  cues.  Design  of  algorithms  to  declare  that  the 
source  has  been  found  using  only  sensed  concentration  is  a  significant  problem  that,  due  to  its 
complexity,  will  be  considered  elsewhere.  The  methodology  used  in  this  article  for  declaration 
that  the  source  has  been  found  is  described  in  the  appendix. 

Due  to  the  turbulent  nature  of  the  fluid  flow,  the  sensed  concentration  is  an  intermittent 
signal,  as  shown  in  Fig.  1.  Fig.  3  depicts  the  situation  and  the  parameters  that  will  be  used  to 
define  practically  the  concept  of  ‘in  the  plume.’  The  top  portion  of  Fig.  3  is  a  schematic  rendition 
of  a  searcher  trajectory  that  finds  the  plume  and  subsequently  makes  along-plume  progress.  The 
lower  portion  of  the  figure  presents  a  magnified  depiction  of  such  an  in-plume  odor  encounter 
scenario.  The  black  dots  indicate  above  threshold  patches  of  odor.  The  time  at  which  the  sensed 
concentration  is  first  above  threshold  is  denoted  7>,„/.  The  searcher  will  be  defined  to  be 


maintaining  contact  with  the  plume  at  time  t,  if  the  sensed  concentration  at  any  time  in  the  last  A 
seconds  has  been  above  threshold  (i.e.,  if  c(  v)>rfor  any  v  e  [t  -A,t  ]).  Note  that  even  while  the 
searcher  is  within  the  time-averaged  boundary  of  the  plume,  there  will  be  periods  where  the 
sensed  concentration  is  below  threshold  (see  also  Fig.  1).  With  reference  to  Fig.  3,  let  TLas, 
denote  the  most  recent  time  that  above  threshold  odor  was  detected.  There  may  be  many  above 
threshold  detections  between  T First  and  Tiast-  Let  Tu>st  denote  Tiasl  +  A.  Note  that  Tiost  is  the  time 
at  which  the  searcher  declares  that  contact  with  the  plume  is  lost.  The  maintain  time  will  be 
calculated  as 

Tm  Tiost  -  TFirst  -  A.  (3) 

Finding  the  plume  expends  both  time  and  energy.  Therefore,  once  the  plume  is  found,  the 

searcher  does  not  want  to  lose  (intermittent)  contact  with  the  plume.  In  addition,  significant 

progress  toward  the  source  can  be  made  once  the  plume  has  been  found.  Therefore,  a  critical 

question  is  how  to  design  algorithms  that  are  effective  at  maintaining  plume  contact  while 

advancing  toward  the  source.  The  nominal  strategy  (motivated  by  studies  of  moth  behavior)  for 

maintaining  contact  with  the  plume  is  to  define  the  searcher  heading3  by 

y/v(t)  =  y/u(t)  +  f3(t)  if  c( v)>tfor  some  v  e  [  t  -A,  t  ])  (4) 

y/v(t)  =  y/u(t)  -  y(t)sign(fi)  if  c( v)>zfor  some  v  e  [  t  -  Tw>  t  -A])  (5) 

where  y/u(t)  =  y/w(t)  +  180  is  the  heading  that  instantaneously  points  upwind  at  the  searcher 

location,  P(t)  has  constant  magnitude  with  a  sign  that  changes  each  time  that  behavior  of  eqn.  (5) 

occurs,  and  yis  a  constant  to  be  defined  later.  This  nominal  behavior  is  illustrated  in  Fig.  3.  If 

c(v)<t  for  all  v  e  [t  -  Tw,  t]),  then  the  searcher  reverts  to  the  plume  finding  behavior  defined  in 


the  appendix. 
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The  performance  level  of  this  strategy  depends  on  environmental  factors  and  the 
parameters  defining  the  plume  maintaining  behavior.  To  account  for  environmental  factors  the 
Monte  Carlo  approach  analyzes  statistical  performance  based  on  numerous  simulations.  Each 
simulation  in  a  Monte  Carlo  batch  uses  a  distinct  instantiation  of  the  plume.  Our  approach  to 
designing  a  plume-maintaining  behavior  includes  evaluation  of  the  effect  of  changing  the 
parameters  of  the  plume-maintaining  strategy.  Strategies  will  be  evaluated  by  the  duration  of  time 
that  they  maintain  contact  with  the  plume.  Possible  alternative  metrics  are  discussed  in  Section  7. 
Increased  maintain  times  are  considered  desirable,  but  in  addition  to  increasing  maintain  time,  it 
is  necessary  to  make  progress  toward  the  odor  source.  The  results  in  Sections  4.2  and  4.3  show 
that  it  is  possible  to  achieve  long  maintain  times  without  making  significant  progress  toward  the 
source.  Section  4.3  presents  an  alternative  strategy  that  is  successful  at  maintaining  contact  with 
the  plume  while  making  significant  progress  toward  the  source.  This  improved  behavioral 
strategy  may  have  biological  counterparts. 

3.1  Maintain  Time  Dependency  on  Waiting  Time  (TV)  and  Threshold  (r) 

For  the  maintain  strategy  of  eqns.  (4-5)  with  J3=  0,  the  sensitivity  of  Tm  to  the  parameters  X 

and  ris  plotted  in  Fig.  4.  Each  iteration  of  the  evaluation  method  consisted  of  100  searchers 
starting  at  (x,y)  —  (85,0)  m.  All  searchers  followed  exactly  the  same  path  during  the  plume¬ 
finding  phase.  Once  the  plume  was  detected,  each  vehicle  used  a  distinct  set  of  X  and  r  values 
for  its  maintain  strategy  until  the  plume  was  lost.  After  all  searchers  lost  the  plume,  the  resulting 
set  of  Tm  values  was  saved.  Then,  the  process  was  repeated,  with  all  100  searchers  restarting  at 
the  same  time  instant  at  (x,y)  =  (85,0)  m.  This  process  was  repeated  for  100  iterations.  The  time- 
varying  nature  of  the  plume  causes  each  of  the  iterations  to  be  distinct.  Reginning  all  100 
searchers  of  each  iteration  at  the  same  time  and  location  with  the  same  plume-finding  behavior, 
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causes  the  evaluation  of  their  performance  to  occur  (for  that  iteration)  under  identical  conditions. 

Decreasing  the  detection  threshold  results  in  increased  Tm  as  shown  in  Fig.  4.  This  effect  is 
expected,  because  decreasing  the  threshold  increases  the  number  of  times  that  odor  is  detected. 
The  plots  showing  that  Tm  increases  with  A  are  more  interesting;  however,  the  result  is  not 
unexpected,  because  increasing  A  gives  the  searcher  additional  time  to  encounter  odor.  What  Fig. 
4  does  not  reveal  is  that  increasing  A  can  result  in  the  searcher  being  significantly  further  from 
the  plume  when  it  decides  that  the  plume  has  been  lost;  this  increased  separation  complicates 
plume  reacquisition.  Note  also  that  Tm  saturates  as  A  increases.  The  results  of  Fig.  4  were  found 
via  Monte  Carlo  simulation  for  a  given  set  of  experimental  conditions.  Changing  the  simulation 
parameters  (experimental  conditions)  changes  the  specific  values  along  the  plots,  but  not  the 
general  trends. 

4  DESIGN  AND  OPTIMIZATION  OF  PLUME  MAINTAINING 

Designing  a  behavior  to  keep  the  searcher  in  contact  with  the  plume  while  making  upwind 
progress,  requires  the  design  of  two  interacting  behaviors:  the  ‘within  the  plume’  (maintaining) 
behavior  and  the  reacquisition  behavior.  The  two  behaviors  are  difficult  to  consider  or  optimize 
independently.  Plume  reacquisition  is  considered  first,  because  those  strategies  are  required  for 
the  subsequent  development  of  plume-maintaining  behavior.  Our  explicit  goal  is  to  define  a 
strategy  of  plume  tracing  for  autonomous  vehicles.  The  process  of  this  design  has  suggested 
principles  relevant  to  plume  following  in  wind  by  flying  and  walking  organisms. 

4.1  Plume  Reacquisition 

The  plume  reacquisition  problem  is  that  A  seconds  after  the  last  above-threshold  detection 
of  odor,  the  vehicle  performs  a  maneuver  intended  to  recontact  the  odor  plume.  If  the  plume  is 
not  recontacted  within  a  time  7V,  then  the  vehicle  reverts  to  plume-finding  behavior.  Because 


plume  finding  is  expensive  in  terms  of  both  time  and  energy,  maximizing  the  probability  that 
plume  reacquisition  succeeds  is  critical. 

The  plume  maintaining  behavior  involves  travel  at  an  angle  / 3  offset  to  upwind  as  indicated 
in  Fig.  3.  For  reacquisition,  a  key  question  is  “when  the  plume  maintaining  strategy  results  in  the 
searcher  losing  contact  with  the  plume,  which  direction  should  the  searcher  turn  to  maximize  the 
likelihood  of  recontacting  the  plume?”  Although  the  obvious  answer  is  to  turn  toward  the  plume, 
this  approach  cannot  be  directly  implemented,  as  the  plume  location  is  unknown  to  the  searcher. 
The  information  available  to  the  searcher  is  the  vehicle  heading  y/v(t),  wind  heading  y/w(t),  sensed 
odor  concentration  c(t),  and  vehicle  position  (x(t),y(t)).  We  study  plume  reacquisition  in  two 
steps.  First,  we  analyze  recovery  time  as  a  function  of  the  recovery  angle  y.  Second,  we  analyze 
the  statistical  relationship  between  measured  infonnation  (the  vehicle  motion  direction  minus  the 
wind  direction)  and  the  optimal  recovery  angles.  The  goal  is  to  determine  a  means  of  using  y/v 
and  \f/w  to  determine  the  optimal  turn  direction  y. 

To  analyze  the  first  issue  a  set  of  simulations  was  run  in  which  each  iteration  involved  36 
vehicles,  indexed  by  the  variable  i  in  [1,  36],  started  at  (x,y)  =  (80,10)  m  and  remaining  at  that 
location  until  odor  was  detected.  After  detection,  each  searcher  traveled  with  the  heading  y/u(t) 
defined  after  eqn.  (5).  Each  searcher  maintains  this  direction  of  travel  until  contact  with  the 
plume  is  lost.  When  the  plume  is  lost,  the  heading  of  the  i-th  vehicle  becomes 

Yv(t)  =  Vu(t)  -yt  + 4  (6) 

where  yplOz  and  <f>  is  a  normal  random  variable  with  a  standard  deviation,  cr.  The  random 

component  <f>  is  inserted  to  allow  analysis  of  the  effect  of  error  in  measuring  wind  heading.  Each 

vehicle  continues  until  either  it  reacquires  the  plume  or  the  simulation  times  out  (T'=200  s).  At 

the  end  of  each  iteration,  the  reacquisition  time  for  each  vehicle  is  saved.  Therefore,  each 


iteration  of  the  above  simulation  results  in  a  record  of  Tr  versus  y.  Each  iteration  was  classified 
into  one  of  two  categories  based  on  this  record.  Category  1  contains  those  records  closest  in 
shape  to  Tr  =  200  for  y  in  [0,180]  and  Tr  =  0  for  y  in  [180,360],  which  corresponds  to  those 
iterations  where  negative  /yielded  better  performance.  Category  2  contains  those  records  closest 
in  shape  to  Tr  —  0  for  y in  [0,180]  and  Tr  =  200  for  yin  [180,360],  which  corresponds  to  those 
iterations  where  positive  /yielded  better  performance.  The  average  of  all  records  in  Category  1  is 
shown  in  Fig.  5a.  The  average  of  all  records  in  Category  2  is  shown  in  Fig.  5b.  This  analysis 
was  repeated  for  three  different  values  of  the  parameter  cr,  with  similar  results. 

Note  that  the  objective  in  this  section  is  to  minimize  recovery  time.  Figs.  5a  and  b  show 
that  \)\  =  90  (i.e.,  crosswind)  provides  both  the  minimal  reacquisition  time  and  the  greatest 
robustness  to  error  (±70°)  in  the  vehicle  heading.  Note  that  at  this  point  in  the  discussion  the 
optimal  magnitude  of  yis  known  and  the  records  have  been  sorted  a  posteriori  according  to  the 
sign  of  ythat  yielded  optimal  performance.  The  remaining  goal  is  to  determine  a  means  for 
reliably  predicting  the  sign  of  y,  during  the  search,  based  on  the  data  available  to  the  searcher 
(i.e.,  y/w(t),  \pv(t),  vehicle  position,  and  c(t)). 

To  solve  this  remaining  problem,  we  further  analyze  the  data  of  the  previous  simulation  for 
the  cases  where  y=-90°  or  y=+90°.  In  this  further  analysis,  we  consider  y/w(t)  and  i//v(t)  at  the 
time  t-TioSt.  For  each  of  the  100  iterations,  we  determine  which  sign  of  /yielded  the  minimum 
value  of  Tr.  For  this  value  of  y  we  further  analyze  i//w(t)  and  y/v(t).  For  the  58  iterations  classified 
as  Category  1  (y=-90°  is  optimal)  90%  had 

Writ)  ~  Wv(0  -  Ww(t)  <  -180°. 

For  the  42  iterations  classified  as  Category  2  (y=+90°  is  optimal)  90%  had 
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*V 

Writ)  =  Vv(t)  -  yjt)  >  -180°. 

These  results  lead  to  the  conclusion  that  by  defining 

7^+90°  if  y/v(t)  -  y/w(t)  >  -180°  and  (7) 

r=  -90°  if  y/v(t)  -  y/w(t)  <  -180°  (8) 

at  /  =  host  the  strategy  should  result  in  successful  plume  reacquisition  with  approximately  90% 

success. 

The  above  results  motivate  Strategy  I  shown  in  Table  1  for  maintaining  plume  contact. 
Two  comments  relative  to  the  physical  interpretation  of  the  results  of  this  section  and  the 
practicality  of  Strategy  1  are  important.  First,  because  Strategy  I  commands  the  vehicle  to  travel 
upwind  (relative  to  the  recent  wind  direction),  the  quantity  y/r(TLost)  =  y/v(TLost)  -  ^w(TLos<)  is 
related  to  the  derivative  of  the  wind  direction.  Therefore,  the  above  conclusions  have  the 
physical  interpretation  that  to  reacquire  contact  with  the  plume,  the  vehicle  should  turn  in  the 
direction  opposite  to  the  change  in  the  wind  direction.  Second,  because  Strategy  I  uses  /?  =  0° 
(i.e.  upwind)  and  the  wind  direction  cannot  be  measured  perfectly,  Strategy  I  may  not  be  robust  in 
the  sense  that  /may  have  the  incorrect  sign  due  to  noise  in  measurement  of  the  wind  direction. 

Fig.  6  shows  a  contour  plot  of  the  source  finding  time  of  Strategy  I,  with  a  vehicle  speed  of 

lms'1,  A  =  1.0  s  and  Tw=  3.0  s  when  combined  with  the  plume  finding  and  source  declaration 

methods  of  the  appendix.  The  data  for  the  contour  plot  is  generated,  with  the  large  meander 
plume  simulation  parameters  (see  Section  2.3),  by  defining  a  regular  10  by  10  grid  of  starting 
locations  over  the  100  by  100  m  search  area.  The  searcher  starts  at  each  location  100  times  (i.e., 
10,000  total  searches).  The  time  to  find  the  source  from  each  starting  location  is  averaged  to 
produce  a  matrix  of  average  times  to  find  the  source  as  a  function  of  position.  In  the  simulations 
corresponding  to  Fig.  6  (and  the  contour  plots  in  later  sections),  if  a  searcher  failed  to  find  the 


source  within  the  300  s  simulation  duration,  then  the  value  300  s  was  used  as  its  source  finding 
time.  The  odor  source  is  located  at  (20,0)  m.  This  form  of  contour  plot  will  be  used  throughout 
the  remainder  of  this  article  to  illustrate  overall  search  performance. 

4.2  Design  of  Maintaining  Behavior  with  Passive  Strategy 

The  previous  subsection  determined  y  based  on  y/r(Tiost)  for  the  case  where  /?=  0.  This 

subsection  extends  the  analysis  to  consider  the  more  general  case  where  /?is  non-zero. 
Throughout  this  subsection,  the  plume  finding  behavior  is  that  described  in  the  appendix.  The 
strategy  for  maintaining  plume  contact  is  as  follows: 

Vv(t)  =  y/u(t)  +  <f>  if  c(t)  >t  (9) 

y/v(t)  =  tf/u(t)  +[}  +  </)  if  c(t)  <  t,  but  c(v)>  t  for  some  v  e  [  t  -X,  t  ]  (10) 

where  for  the  z'-th  vehicle  P  =-60+120/736,  i  is  the  vehicle  index  in  [1,36],  and  <f>  is  a  uniform 
random  variable  with  maximum  amplitude  A.  With  this  strategy,  the  searcher  turns  toward 
upwind  while  sensing  above  threshold  concentration  and  turns  to  an  angle  P  relative  to  upwind 
when  not  instantaneously  detecting  above  threshold  odor.  If  the  vehicle  has  not  detected  above 
threshold  odor  within  the  last  X  seconds,  then  that  iteration  of  that  vehicle’s  simulation  ends. 

For  each  iteration  of  the  simulation  36  vehicles  started  at  (x,y)  =  (85,0)  with  velocity  1.0  m 
s'1.  All  36  vehicles  implemented  the  same  plume  finding  behavior  until  odor  was  detected.  After 
odor  detection  all  vehicles  implement  the  strategy  described  above  with  a  different  value  of  /?  for 
each  vehicle.  Each  vehicle  continues  until  either  the  simulation  times  out,  odor  is  not  detected 
for  X  seconds,  or  the  source  was  found.  For  each  searcher  and  each  iteration,  the  total  amount  of 
time  in  contact  with  the  plume  (i.e.,  the  time  duration  that  (9)  and  (10)  are  used)  and  i//r(Ti0S()  = 
y/w(T Lost)  -  Vv(TLost)  are  recorded.  This  process  was  repeated  for  100  iterations. 


Each  iteration  of  the  above  simulation  results  in  a  record  of  Tm  versus  J3.  Each  iteration 
was  classified  into  one  of  two  categories  by  a  process  similar  to  that  described  in  the  previous 
section.  Category  1  corresponds  to  those  iterations  where  a  positive  J3  yielded  better  performance. 
Category  2  corresponds  to  those  iterations  where  a  negative  J3  yielded  better  performance.  The 
average  of  all  records  in  Category  1  is  show  in  Fig.  7a.  The  average  of  all  records  in  Category  2 
is  show  in  Fig.  7b.  The  batch  of  100  iterations  was  repeated  twice,  once  with  the  random 
component  <f>  equal  to  zero  and  once  with  the  random  component  uniformly  distributed  with  (j>  e 
[-10°,  10°]. 

For  the  Category  1  iterations,  y/r(TLost)  >-180°  degrees  for  81%  of  iterations  when  ^=0° 
and  78%  of  iterations  when  <f>\s  uniform  over  ±10°.  For  the  Category  2  iterations,  y/r(TLost)  <-180 
°  for  91%  of  iterations  when  ^  =  0°  and  86%  of  iterations  when  <f>  is  uniform  over  ±10°. 
Combining  these  statistical  relations  with  the  conclusion  of  the  previous  section,  we  conclude 
that  we  can  use  signf/?)  to  determine  the  sign  of  /instead  of  using  y/r(TL0St).  This  is  useful  since 
J3  is  an  internal  variable  whereas  y/r(TLost)  is  a  possibly  noisy  measurement.  The  resulting 
strategy  is  referred  to  as  Strategy  n,  and  is  summarized  in  Table  2.  Note  that  Strategy  II  is  a 
countertuming  strategy.  The  reacquisition  behavior  requires  the  searcher  to  travel  in  the  opposite 
crosswind  direction  to  the  maintain  behavior.  Fig.  7  shows  that  any  /?>  10°  yields  good 
performance.  Improving  performance  by  allowing  /3  to  be  a  time-varying  function  will  be 
considered  in  the  next  section. 

It  is  interesting  to  compare,  for  a  biological  entity,  the  two  strategies  of  (i)  traveling  with 
/?  =  0  and  attempting  to  reacquire  based  on  y/r(T LoSt),  or  (ii)  traveling  with  /?*  0  and 
countertuming  based  on  sign(/7).  Strategy  I  requires  a  low  noise  sensor  with  a  bandwidth  high 


enough  to  rapidly  and  accurately  detect  y/r(TLoS,).  Note  that  with  /?=  0  accurate  decisions 
concerning  whether  y/r(Ti0St)  is  greater  or  less  than  180°  are  not  robust  to  wind  sensor  noise.  For 
biological  flying  entities,  a  combination  of  airspeed  and  visual  cues  could  yield  a  low  bandwidth 
wind  direction  estimate  (see  discussion  in  Carde  and  Knols,  2000),  but  this  system  of 
measurement  is  unlikely  to  produce  an  accurate  high  bandwidth  estimate  of  y/r(T Lost).  Strategy  II 
requires  only  a  low  bandwidth  (relative  to  the  vehicle  dynamics)  wind  sensor  to  maintain  the 
wind  relative  direction  of  travel.  The  countertuming  of  Strategy  II  is  inherently  more  robust  to 
wind  sensor  noise,  since  the  countertuming  direction  is  not  directly  dependent  on  instantaneously 
sensed  wind  direction.  A  disadvantage  of  Strategy  II  is  that  its  mean  ‘in  the  plume’  path  to  the 
source  will  be  longer  than  that  of  Strategy  I  since  for  Strategy  II  (3*  0.  Strategy  I,  which  has 
similarities  with  aim-then-shoot  strategies,  described  by  Dindonis  and  Miller  (1980)  and  named 
by  Kennedy  (1986),  is  advantageous  either  when  the  flow  field  is  nearly  uniform  and  time 
invariant  or  when  the  searcher  is  very  near  the  source  where  the  wind  direction  is  reasonably  well 
correlated  with  the  direction  of  the  plume’s  long  axis;  however,  when  Strategy  I  causes  the 
searcher  to  exit  the  plume  the  searcher  may  not  be  able  to  turn  reliably  in  the  correct  direction  to 
rapidly  reacquire  the  plume.  Strategy  II  has  a  plume  reacquisition  advantage,  which  is  beneficial 
when  the  searcher  is  following  long,  meandering  plumes. 

A  physical  interpretation  for  the  utility  of  ft  *  0  for  predicting  the  correct  direction  to  turn 
to  reacquire  the  plume  is  that  if  the  vehicle  travels  due  upwind  within  the  plume  (i.e.,  /?  =  0),  then 
at  the  moment  when  the  vehicle  exits  the  plume  the  vehicle  is  equally  likely  to  be  on  either  side 
of  the  plume.  Therefore,  without  additional  information,  either  direction  the  vehicle  turns  has  an 
equal  chance  of  immediately  re-contacting  the  plume.  Alternatively,  if  the  vehicle  travels  to  the 
left  of  upwind  (J3  <  0)  while  in  the  plume,  the  vehicle  is  more  likely  to  exit  the  plume  via  its  left 


edge.  Therefore,  a  right  turn  (counterturn)  improves  the  likelihood  of  quickly  re-contacting  the 
plume. 

Note  that  it  may  be  possible  to  increase  the  perfonnance  by  using  both  the  information  y/r 
and  crosswind  travel  {fi  *  0).  We  have  not  pursued  this  line  of  analysis  since  the  increased 
distance  of  travel  is  not  large  if  {fi  <45°)  and  we  do  not  expect  y/r  to  be  measurable  with  high 
resolution  or  bandwidth  on  autonomous  vehicles.  Also,  our  experience  in  this  project  showed 
that  failure  of  the  plume  reacquisition  process  resulted  in  a  large  time  penalty,  whereas  crosswind 
travel  resulted  in  only  a  small  penalty  (for  fi<A5°).  The  active  strategy  of  the  next  section  tries  to 
balance  these  two  issues  by  making  upwind  progress  (fi  near  0)  when  entering  the  plume  and 
increasing  the  likelihood  of  countertuming  success  by  crosswind  travel  {fi  approaching  90°)  as 
the  time  in  the  plume  increases. 


4.3  Design  of  Maintaining  Behavior  with  Active  Strategy 

The  previous  section  demonstrated  the  advantage  of  the  countertuming  behavior  with 

fi*  0.  In  this  section,  we  analyze  the  possible  benefits  of  using  a  time-varying  upwind  angle  fi(i) 

to  improve  the  tradeoff  between  up-plume  progress  and  maintaining  contact  with  the  plume.  In 

this  ‘active’  strategy,  the  vehicle  heading,  within  the  plume,  will  be 

Vv(t)  =  y/u(t)  +  <f>  +  P(t),  (11) 

where  (f>  is  an  error  component  simulating  sensor  inaccuracy  and 


f  ft  {t~T First)  1 
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rl  <  ft  -  T First )  *  T2 


3ft  (t  Tpjrst )  >  T2 


is  a  time  varying  offset  to  upwind  with  Tpirst  redefined  after  each  plume  reacquisition. 


The  time  variation  is  illustrated  in  Fig.  8.  Upon  first  entering  the  plume,  the  searcher  turns 
toward  y/v  =  y/u(t)  (i.e.,  P  small).  As  the  time  in  the  plume  increases,  P(t)  increases  in  magnitude 
(toward  either  ±90°).  The  sign  of  P  is  selected  based  on  the  direction  the  searcher  was  moving 
upon  entering  the  plume,  so  that  the  vehicle  continues  progress  into  the  plume.  Therefore,  on  first 
entering  the  plume  (i.e.,  P(t)  near  0)  significant  up-plume  progress  is  achieved.  As  \P(t)\ 
increases,  the  searcher  traveling  across  the  plume  will,  by  design,  leave  the  plume.  Since  the 
direction  of  cross  plume  travel  is  known  at  Tiost,  the  searcher  knows  the  direction  in  which  to 
turn  (from  the  results  of  the  previous  section)  to  maximize  the  likelihood  of  rapidly  reacquiring 
the  plume.  Upon  reacquisition,  the  time  varying  P(t)  process  is  repeated  with  the  opposite  signs. 

Optimization  of  this  strategy  requires  analysis  of  the  effects  of  various  parameters.  To 
analyze  the  effects  of  the  strategy  parameters,  100  iterations  of  the  following  set  of  simulations 
were  performed.  For  each  iteration,  100  searchers  were  initiated  at  (x,y)  =  (60,5)  m.  All  100 
searchers  performed  exactly  the  same  plume  finding  maneuvers  until  odor  was  detected.  At  the 
time  of  detection,  each  searcher  initiated  the  plume  maintaining  behavior,  with  a  distinct  set  of 
strategy  parameters.  For  example,  for  the  left  column  of  plots  in  Fig.  9,  in  the  approaches  using 
constant  p  (i.e.,  the  bottom  curve),  the  i-th  vehicle  used  P  =  0.85/.  Each  iteration  results  in  a 
record  of  Tm  versus  p.  The  100  records  from  the  100  iterations  are  averaged  to  produce  the  lower 
curve  of  the  left  column  of  Fig.  9.  For  this  study,  only  one  parameter  was  changed  at  a  time.  For 
all  the  simulations,  the  reacquisition  strategy  parameters  of  the  previous  section  were  y=±90°  and 
X  =  1.0  s.  If  the  searcher  reacquired  the  plume  within  X  seconds,  then  Tm  kept  accumulating. 
Otherwise,  the  simulation  of  that  searcher  and  that  iteration  terminated. 

The  left  column  of  graphs  in  Fig.  9  evaluates  the  effect  of  the  search  angle  parameter  /?. 
The  top  left  graphs  shows  the  results  of  simulations  using  the  set  of  simulation  parameters  that 
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results  in  large  scale  plume  meander.  The  bottom  left  graphs  shows  the  results  of  simulations 
using  the  set  of  simulation  parameters  that  results  in  small  scale  plume  meander.  Note  that  the 
maintain  time  is  significantly  longer  when  the  meander  is  small.  The  bottom  curve  of  each  of  the 
left  column  graphs  indicates  average  maintain  time  as  a  function  of  the  constant  angle  fie  [0°,75° 
],  as  shown  in  Fig.  3.  This  curve  shows  that  maintain  time  is  increased  by  traveling  at  larger 
crosswind  angles;  however,  larger  crosswind  angles  result  in  less  up-plume  progress  per  transect. 
The  top  curve  of  each  of  the  left  column  graphs  indicates  average  maintain  time  as  a  function  of 
the  time  varying  angle  fi(r),  as  shown  in  Fig.  8  and  defined  following  eqn.  (11).  To  produce  the 
fi(t)  curves,  fij  (=0.25i  for  the  i-th  vehicle)  varied  (in  different  batch  simulations)  from  0°  to  25°. 
Therefore,  3  fii  which  is  the  maximum  of  fi(t)  was  varied  from  0°  to  75°.  The  left  set  of  graphs 
shows  that  there  is  a  significant  advantage  in  terms  of  time  in  the  plume  when  using  the  time- 
varying  fi  approach.  In  addition,  the  approach  makes  substantial  up-plume  progress  each  time  the 
searcher  re-enters  the  plume  as  fi(t)  reinitializes  to  near  0°.  The  right  column  of  graphs  in  Fig.  9 
show  the  effect  of  searcher  speed  on  TM.  The  top  graph  shows  TM  versus  speed  of  travel  during 
reacquisition.  The  bottom  graph  shows  Tm  versus  speed  of  travel  within  the  plume.  For  both 
graphs,  the  horizontal  axis  is  the  multiple  of  wind  speed  N  (i.e.,  Vehicle  speed  equals  N  times  the 
average  wind  speed.).  The  function 
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.2s  <  (t  —  Tfirst )  <  0.5s 

(12) 

85 

(t  —  Tpirst )  >  0.5s 

with  T First  redefined  after  each  plume  reacquisition,  was  used  during  the  simulations  that 
generated  the  right  column  of  Fig.  9  and  will  be  used  in  all  subsequent  simulations. 
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4.4  Strategy  Summary 

Sections  4. 1  -  4.3  have  developed  alternative  strategies  for  maintaining  contact  with  an 
odor  plume.  These  strategies  are  summarized  in  Table  3.  The  results  of  Section  4.1  showed  that 
countertuming  behavior  was  beneficial  to  the  process  of  reacquiring  the  plume,  but  used  a 
mechanism  (difference  between  the  wind  and  vehicle  directions)  that  was  not  robust  to  wind 
sensor  noise.  Section  4.2  showed  that  crosswind  travel  while  in  the  plume  simplifies  the 
definition  of  the  countertum  and  makes  that  definition  relatively  immune  to  wind  sensor  noise. 
Section  4.2  improved  the  rate  of  travel  up  the  plume  by  varying  the  angle  of  crosswind  travel 
while  within  the  plume. 

The  relative  performance  of  the  three  strategies  under  identical  conditions  is  shown  in  Fig. 
10.  This  figure  was  generated  by  starting  searchers  using  each  strategy  at  positions  along  the  x- 
axis  (i.e.,  (z',0),  i=  0,  ...,  100).  Each  strategy  was  started  500  times  from  each  starting  location. 
The  average  time  to  find  the  source  versus  starting  location  for  each  strategy  is  shown  in  Figure 
10.  For  initial  locations  far  from  the  odor  source,  Strategy  m  yields  30  to  40  seconds  of 
improvement  over  Strategy  I  and  10  seconds  of  improvement  over  Strategy  H  Near  the  odor 
source,  where  the  wind  and  plume  axis  are  nearly  in  the  same  direction,  all  three  strategies  have 
nearly  identical  performance. 

The  comparison  between  the  strategies  of  Table  3  and  the  behavior  of  flying  moths  and 
other  biological  entities  is  not  necessarily  straightforward  for  the  following  reasons.  For  the 
strategies  developed  herein,  the  behaviors  are  explicitly  known,  because  they  are  programmed. 
For  moths,  the  behaviors  are  known,  but  the  sensory  inputs  are  inferred,  based  on  observations  of 
moths  flying  in  wind  tunnels  where  only  the  approximate  instantaneous  position  of  the  plume  is 
known  (e.g.,  Justus,  Schofield,  Murlis  and  Carde,  2002).  For  the  simulation  analyses  herein,  the 


method  allows  explicit  knowledge  of  when  the  searcher  is  detecting  odor.  Finally,  the 
simulations  involve  a  large  search  area  and  plumes  with  significant  meander.  Much  of  the 
literature  on  biological  plume  tracing  involves  biological  entities  confined  within  a  flume  or 
wind  tunnel.  Such  experiments  necessarily  are  restricted  to  a  relatively  small  search  area  within 
which  plume  meander  is  constrained  by  the  vertical  and  lateral  expanse  of  the  wind  tunnel  or 
flume.  Such  wind  tunnel  experiments  correspond  to  the  portion  of  Fig.  10  where  all  three 
strategies  yield  nearly  identical  levels  of  performance. 

5  PERFORMANCE  EVALUATIONS  USING  SIMULATED-ODOR  PLUMES  AND 
EXPERIMENTAL  WATER  FLUMES 

This  section  presents  performance  evaluation  for  the  complete  Strategy  El  (including 
plume  finding  and  source  declaration  as  described  in  the  Appendix)  with  /?  defined  in  eqn.  (12), 
relative  to  the  simulated  plume  and  three  sets  of  experimental  flume  data.  During  the  simulation 
evaluation,  the  measured  fluid  flow  was  corrupted  by  additive  noise  <|>  that  is  a  white  normal 
random  process  that  changes  value  at  a  1.0  Hz  rate.  If  c(v)<z for  all  ve  [  t-Tw,  t  -A],  then  the 
vehicle  reverts  to  the  plume-finding  behavior. 

Four  sets  of  batch  simulations  were  evaluated.  For  each  batch  evaluation,  the  strategy  and 
vehicle  model  were  identical,  but  the  source  of  the  concentration  and  flow  data  were  unique. 
Four  sources  of  concentration  and  flow  data  were  used:  the  simulation  model,  flume  data  from 
Stanford  University  (Crimaldi  and  Koseff,  2001),  Cornell  University  (Cowen,  Change,  and  Liao, 
2001),  and  Georgia  Institute  of  Technology  (Webster,  Roberts,  and  Ra’ad,  2000;  Webster  and 
Weissburg,  2001).  Each  of  the  flume  data  sets  contained  both  concentration  and  flow  information 
obtained  by  exciting  a  fluorescent  dye  with  a  laser  sheet  within  a  portion  of  a  flume.  Images  of 
this  area  were  obtained  by  video  camera  and  stored.  The  image  intensity  is  interpreted  as 


concentration.  Correlation  of  concentration  data  between  sequential  images  was  used  to 
determine  the  local  flow.  Interpolation  between  images  based  on  the  fluid  velocity  was  used  to 
generated  plumes  changing  continuously  in  time  from  the  discrete  images.  The  image,  flow 
vector,  and  time  were  scaled  so  that  the  image  fit  a  20  by  60  m  portion  of  the  100  by  100  m 
search  area.  The  20  by  60  m  flume  image  was  placed  within  the  search  area  so  that  the  source 
appeared  to  be  at  (20,0)  m.  Outside  the  flume  image  area,  the  concentration  was  set  equal  to  zero 
and  the  flow  was  equal  to  the  mean  flow  over  the  flume  image. 

Fig.  11  shows  contour  plots  of  the  source  finding  time  of  Strategy  El  with  X  =  1 .0  s  and  7V 
=  4.0  s.  Fig.  11  contains  data  from  four  different  batches  of  simulations.  The  data  for  each 
contour  plot  is  generated  by  defining  a  10  by  10  grid  of  starting  locations  over  the  100  by  100  m 
search  area.  The  searcher  starts  at  each  location  100  times  (i.e.,  10,000  total  searches).  The  time 
to  find  the  source  from  each  starting  location  is  averaged  to  produce  a  10  by  10  matrix  of  average 
time  to  find  the  source  as  a  function  of  position.  The  odor  source  is  located  at  (20,0)  m.  For  the 
upper  left  contour  plot,  the  large  meander  plume  simulation  was  used.  Proceeding  clockwise,  the 
contour  plots  were  generated  using  experimental  data  sets  from  Stanford,  Georgia  Tech,  and 
Cornell.  Fig.  1 1  should  be  compared  with  Fig.  6  to  observed  the  improved  performance  achieve 
by  optimization  of  the  plume  maintaining  and  reacquisition  behaviors.  Note,  by  comparison  of 
the  four  subfigures  of  Fig.  11,  that  performance  is  least  proficient  for  the  simulated  plume.  This 
is  caused  by  the  flumes  lacking  significant  lateral  meander  due  to  the  width  constraint  of  the 
flume.  Therefore,  the  simulated  plume,  with  the  large  lateral  meander  typical  of  plumes  in 
natural  environments,  is  significantly  more  challenging  to  trace  than  (currently  available)  wind 
tunnel  or  flume-generated  experimental  data  sets.  Also,  note  that  97%  of  all  searchers  succeeded 
in  locating  the  source  within  aim  radius  within  the  300  s  time  limit. 


6  PLUME  FOLLOWING  STRATEGIES  OF  ANIMATS  AND  BIOLOGICAL  EXEMPLARS 

The  selective  pressures  for  optimizing  strategies  of  plume  following  can  be  expected  to 
vary  with  the  adaptive  value  of  rapid  location  the  odor’s  source,  and  with  the  penalty  for 
particular  plume-following  strategies  that  result  in  loss  of  the  scent.  It  may  be  that  maneuvers 
such  as  high  upwind  velocity  that  seemingly  promote  rapid  location  are  also  more  prone  to  cause 
the  searcher  to  exit  and  possible  loose  the  plume.  Plume  following  by  flying  male  moths  seeking 
a  pheromone-emitting  female  can  be  expected  to  be  highly  constrained  to  behaviors  that  promote 
rapid  location  of  the  female,  given  that  the  first-arriving  male  is  generally  most  apt  to  mate  with 
the  female  (Carde,  1986).  In  contrast,  location  of  a  host  plant  by  tracing  its  odor  plume  typically 
may  not  be  so  time  constrained  in  that  the  plant  remains  as  an  available  resource  until  it  is 
located.  Finding  a  prospective  plant  host  after  several  minutes  of  searching  is  not  as  likely  to 
have  as  strong  a  selective  disadvantage  as  failing  to  find  a  mate. 

A  feature  common  to  moths  maneuvering  along  a  pheromone  plume  is  a  zigzagging4  path. 
In  our  simulation  such  countertuming  (/?*  0)  would  seem  to  have  the  following  advantages: 
increased  ability  to  maintain  intermittent  contact  with  the  plume;  and,  decreased  sensitivity  to  the 
sensed  wind  direction.  In  moths,  countertuming  usually  is  assumed  to  reflect  an  underlying 
motor  program  that  is  suppressed  when  the  trajectory  is  aimed  more  directly  upwind  (Baker, 
1990).  The  importance  of  countertuming  (producing  a  generally  zigzag  path)  to  the  success  of 
upwind  orientation  has  not  been  fully  resolved.  Preiss  and  Kramer  (1986)  proposed  that  a  zigzag 
path  was  simply  a  consequence  of  moths  being  unable  to  gauge  upwind  direction  accurately; 
therefore,  reiterative  corrections  toward  upwind  yielded  the  countertuming.  The  fact  that  some 
moths  can  fly  upwind  without  evident  countertums  (Carde  and  Mafra-Neto,  1996),  however, 
argues  against  this  explanation.  Zigzagging  also  can  be  viewed  as  a  strategy  to  enhance  the 


information  available  to  gauge  upwind  direction  (Carde,  1984):  moths  heading  off  the  windline 
experience  more  transverse  image  flow,  the  visual  cue  used  to  set  their  heading.  Zigzagging  as 
well  enables  the  moth  to  sample  a  wider  expanse  of  the  plume,  which  might  also  increase  the 
male’s  ability  to  trace  the  plume  (Kennedy,  1983).  Although  a  countertum  generally  occurs  when 
a  moth  ‘exits  the  plume,’  and  therefore  brings  the  moth  back  within  the  plume,  countertums  also 
can  occur  within  the  plume’s  boundaries  (Kennedy  et  al.,  1980;  Willis  and  Baker,  1984;  Justus 
and  Carde,  2002),  and  therefore  their  occurrence  and  temporal  regularity  seem  dictated  by  a 
putative  countertum  generator. 

The  sensory  inputs  and  causal  mechanisms  governing  countertuming  remain  unverified. 
The  observer  of  the  experiment  specifies  when  the  moth  is  ‘in  the  plume.’  In  some  cases,  the 
intermittent  nature  of  the  plume  could  be  causing  the  moth’s  sensed  concentration  to  be  on 
occasion  below  threshold  even  when  the  experimental  observer  declares  that  the  moth  is  ‘in  the 
plume.’  Countertuming  need  not  be  mandated  by  an  internal  motor  program,  but  rather  it  could 
be  dictated  by  loss  of  the  plume  followed  by  a  turn  back  toward  the  direction  where  it  was  last 
sensed  (as  proposed  explicitly  by  Kennedy,  1977,  cf.  Kennedy,  1983).  Even  though  there  is 
debate  over  the  neurobiological  source  of  countertuming  (i.e.,  a  countertum  generator)  and  the 
ethological  mechanisms  (induced  by  a  loss  of  scent  or  a  corrective  navigational  maneuver),  a 
main  conclusion  of  these  simulations  is  that  there  is  an  advantage  to  countertuming  for  robustly 
maintaining  contact  with  a  plume. 

The  strategies  used  by  male  moths  flying  upwind  along  a  pheromone  plume  also  include 
linear  surges  aligned  with  the  instantaneous  wind  flow  and  a  zigzag  trajectory  with  countertums 
typically  expressed  several  times  a  second.  In  the  almond  moth,  Cadra  cautella,  a  continuum 
between  both  forms  of  these  flight  tracks  is  expressed,  contingent  upon  the  temporal  pattern  of 


the  moth’s  contact  with  filaments  of  pheromone  (Mafra-Neto  and  Carde,  1994).  Countertuming 
with  little  or  no  progress  upwind  is  caused  by  sporadic  encounters  with  odor  filaments,  usually  at 
a  rate  <5  Hz,  and  a  zigzag  path  upwind  is  evoked  by  rates  >5  Hz.  In  some  moths,  rates  above  5 
Hz  produce  an  accelerated  velocity  and  a  straightened-out  track  (Justus  et  al.,  2002).  In  such 
wind  tunnel  assays,  fast,  relatively  straight  tracks  are  more  apt  to  result  in  location  of  the  odor 
sources,  and  to  decease  the  time  spent  in  orientation  (Mafra-Neto  and  Carde,  1994). 

The  simulation  Strategy  I— heading  due  or  nearly  due  upwind  when  odor  has  been  recently 
sensed— matches  the  behavior  of  some  moths  encountering  filaments  of  pheromone  at  rates  of  at 
least  5  Hz  (e.g.,  Mafra-Neto  and  Carde,  1994;  Vickers  and  Baker,  1994a).  In  the  context  of  a 
wind  tunnel  assay  over  a  distance  of  less  than  2  meters  with  an  unvarying  wind  velocity  and 
direction,  this  type  of  strategy  enhanced  the  likelihood  and  rapidity  of  finding  a  pheromone 
source.  In  Strategies  II  and  in  the  track  is  angled  upwind  while  contact  with  the  plume  is 
maintained;  turns  occur  when  the  search  exits  the  plume.  The  direction  of  the  turn  is  back 
toward  the  plume’s  putative  centerline  and  toward  the  windline.  These  two  strategies 
superficially  resemble  the  zigzag  path  assumed  by  many  moths  flying  along  a  pheromone  plume 
(Carde,  1996)  and  other  insects  walking  upwind  in  the  presence  of  pheromone  (Bell,  1984). 

There  are  two  important  distinctions.  In  many  wind  tunnel  assays  in  which  the  boundaries 
of  the  plume  are  visually  defined  by  the  observer,  a  moth  will  perform  some  countertums  while 
within  the  plume’s  boundaries  (e.g.,  Kennedy  et  al.,  1980;  Willis  and  Baker,  1984),  a  response 
not  dictated  by  the  Strategies  II  and  III;  however,  this  behavior  also  could  result  if  the 
countertum  was  triggered  by  plume  intermittency  even  while  the  searcher  was  supposed  to  be  ‘in 
the  plume.  ’ 

Another  issue  is  the  spatial  and  temporal  scale  of  wind-tunnel  assays  of  moth  trajectory. 


Our  model’s  searcher  velocity  is  set  at  1  m  s'1  with  a  simulation  step  size  of  0.01  s;  therefore,  the 
functional  step  size  of  the  simulation  is  0.01  m  every  0.01s.  Wind  tunnel  assays  are  highly 
constrained  by  the  wind  tunnel’s  dimensions  (which  rarely  exceed  3  m  in  length  and  1  m  in 
width),  and  movements  are  monitored  many  times  a  second  (e.g.,  0.03  s’1).  However,  in  nearly 
all  wind  tunnel  assays  the  plume  is  filamentous,  so  that  the  odor  signal  flickers  many  times  a 
second  (Justus  et  al.,  2002).  In  such  plumes,  males  may  countertum  several  times  per  second.  In 
experimental  manipulations  of  the  plume  structure  in  a  wind  tunnel,  the  spatial  scale  over  which 
the  plume  was  generated  could  have  contributed  to  the  finding  that  the  moths  seeming  to  use 
Strategy  I  had  a  clear  advantage  over  those  seeming  to  use  Strategies  II  and  III,  in  contrast  to  our 
simulations.  In  any  case,  the  size  of  the  wind  tunnel  did  not  allow  observations  of  more  than  4  s 
of  flight  over  more  than  1-2  m  and  so  the  applicability  of  these  observations  to  plume  following 
in  the  field  cannot  yet  be  assessed. 

The  spatial  and  temporal  scale  at  which  our  simulation  is  defined  also  may  explain  why  the 
highly  successful  Strategy  HI  strategy  in  our  simulations  does  not  appear  to  have  an  exact 
counterpart  observed  to  date  in  flying  moths,  or  in  the  reactions  of  other  flying  or  walking  insects 
navigating  along  an  odor  plume  in  wind  tunnels.  Not  unexpectedly,  wind  tunnel  studies  in 
general  seem  to  simulate  plume  structure  conditions  found  near  the  source:  a  narrow  plume  and 
an  alignment  of  the  plume’s  long  axis  and  the  windline.  Strategies  II  and  HI  implement  a 
countertuming  behavior  that  requires  only  a  low  bandwidth  measurement  of  the  wind  direction 
and  a  binary  concentration  sensor.  Currently,  it  is  not  clear  whether  similar  behaviors  in 
biological  entities  are  the  result  of  the  entity’s  limited  sensing  ability  (e.g.,  sensing  wind  direction 
to  20°,  see  Preiss  and  Kramer,  1986)  or  whether  these  behaviors  are  the  result  of  the  evolutionary 
optimization  process,  because  they  give  the  searcher  some  advantage  in  the  task  of  tracing  the 


plume.  Some  speculations  based  on  the  analysis  herein  have  been  included.  Although  the 
methods  of  this  article  provide  no  evidence  of  how  these  behaviors  might  be  implemented 
biologically,  the  studies  of  this  article  demonstrate  that  the  countertuming  behavior  with  at  least 
15°  deflection  from  upwind  (Strategy  III)  provides  a  significant  advantage  to  behaviors  that 
maintain  a  constant,  nearly  due  upwind,  direction  within  the  plume.  Strategy  HI  may  be  most 
valuable  at  moderate  to  long  distances  from  the  odor  source  where  the  local  wind  vector  either 
does  not  align  with  the  plume’s  centerline  or  point  toward  the  odor  source.  This  is  exemplified 
in  Fig.  10:  Strategies  I,  II  and  HI  yield  equivalent  performance  within  about  5  m  of  the  source. 

Other  efforts  to  model  plume  following  tactics  have  not  addressed  the  range  of  issues 
considered  in  our  simulations.  Williams  (1994)  modeled  tsetse  fly  seeking  of  traps  baited  with 
host  odor,  comparing  the  rapidity  and  probability  of  source  location  by  upwind,  anemotactic 
flight  while  in  the  plume.  Location  of  the  odor  source  was  essentially  contingent  on  how 
accurately  flies  were  able  to  detect  wind  direction,  in  effect  to  bias  their  2-D  ‘random-walk’ 
toward  upwind.  The  plume’s  fine-scale  structure  and  plume  meander  were  not  explicit 
components  of  the  Williams’  model. 

Kramer  (1996)  has  provided  another  approach,  modeling  the  upwind  walking  behavior  of 
the  commercial  silkworm  moth  ( Bombyx  mori )  and  a  hypothetical  flying  moth.  Within  the 
plume’s  overall  boundaries,  the  plume  was  assumed  to  have  an  internal  filamentous  structure 
(with  a  mean  pulse  interval  of  0.33  s)  that  induced  upwind  displacement.  The  precision  of 
upwind  heading  was  set  to  improve  when  the  odor-pulse  rate  increased  up  to  3-5  s'1,  whereas  at 
higher  rates  upwind  displacement  ceased.  Countertuming  was  governed  by  a  small  internal 
tendency  to  turn,  the  sign  of  which  “flip-flops,”  producing  an  upwind  zigzag  and  casting.  The 
task  set  forth  in  these  models  was  to  demonstrate  that  these  design  features  would  yield  tracks 


that  resembled  the  actual  walking  and  flight  tracks  and  to  suggest  how  a  modulation  of  “internal 
turning  tendency”  might  alter  the  success  of  orientation.  Actual  odor  plumes  in  wind  have  much 
high  rates  of  filament  production  than  5  s'1  (Murlis  et  al,  2000). 

Belanger  (Belanger  and  Willis  1996a,  1996b,  1998;  Belanger  and  Arbas,  1998),  who  also 
perform  simulation  studies  motivated  by  moth  behavior,  suggest  strategies  with  similar 
underlying  ideas  to  those  proposed  herein.  A  short  description  of  their  Surger  strategy  (see  pp 
355-356  in  Belanger  and  Arbas  (1998)  for  a  detailed  discussion)  is  that  recent  odor  detection 
causes  the  searcher  to  move  upwind  at  a  commanded  wind  relative  course  generated  once  per 
detection  (after  accounting  for  latencies  and  olfactory  refraction)  with  mean  0°  and  standard 
deviation  40°.  If  odor  detection  does  not  occur  before  an  internal  timer  achieves  a  threshold,  then 
casting  flight  begins.  Casting  flight  continues  until  either  odor  is  detected  or  the  simulation  times 
out.  Their  Surger  strategy  has  some  similarities  with  Strategies  I  and  II  proposed  herein.  Note 
however  that  the  Surger  strategy  does  not  force  the  surges  to  alter  their  crosswind  relative 
direction.  Strategies  I  and  II  proposed  herein  both  attempt  to  force  countertuming  after  contact 
with  the  plume  is  lost.  The  Belanger  Countertumer  strategy  (see  pp  353-354  in  Belanger  and 
Arbas  (1998)  for  a  detailed  discussion)  generates  countertums  when  the  time  since  the  last  turn 
exceeds  a  time-varying  threshold.  If  odor  is  detected  within  the  previous  oscillator  cycle,  then 
the  commanded  course  angle  is  generated  once  per  oscillator  cycle  as  a  unifonn  random  variable 
with  magnitude  in  [0°,70°]  and  sign  defined  to  enforce  a  countertum.  If  odor  is  not  detected 
during  the  previous  oscillator  cycle,  then  the  commanded  course  angle  is  generated  once  per 
oscillator  cycle  as  a  uniform  random  variable  with  magnitude  in  [85°,  105°]  and  sign  defined  to 
enforce  a  countertum.  This  Countertumer  strategy  has  similarities  with  Strategy  III  proposed 
herein.  The  main  differences  are  the  countertums  of  Strategy  ID  are  triggered  by  lack  of  recent 


odor  detection  whereas  those  of  Belanger  are  triggered  by  an  internal  clock.  Also,  after 
detection,  the  course  commands  of  Belanger  are  defined  to  be  upwind  plus  a  random  component. 
In  Strategy  El,  the  after-detection  course  commands  initially  point  toward  upwind,  but  vary  with 
time  toward  crosswind  to  purposefully  increase  the  likelihood  that  the  searcher  knows  from 
which  plume  edge  it  has  departed.5 

This  article  and  the  studies  of  Belanger  et  al.  have  different  objectives,  which  result  in 
different  methods  of  analysis.  The  goal  of  Belanger  et  al.  was  “to  determine  whether  any  of  the 
models  in  the  literature  to  account  for  moth  orientation... could  reproduce  the  behavior... in  M 
sexta.”  Therefore,  they  faithfully  attempted  to  implement  such  models  in  software  and  were  only 
interested  in  biologically  feasible  model  parameters.  Therefore,  model  parameters  were  based  on 
those  in  the  literature  and  numerous  simulations  were  performed  both  to  analyze  performance 
sensitivity  to  parameter  settings  and  to  evaluate  model  performance  relative  to  moth  wind  tunnel 
experiments.  Alternatively,  our  goal  was  to  start  with  models  based  on  moth  behaviors  and 
develop  them  into  good  algorithms  for  autonomous  vehicles.  Our  simulations  were  focused  on 
the  design  process  of  analyzing  good  parameter  settings  and  of  identifying  structural 
improvements  in  the  strategies  that  could  significantly  improve  performance.  The  fact  that  at  the 
end  of  this  process  Strategy  El  and  the  parameters  defined  herein  are  biologically  relevant  is 
noteworthy. 

The  different  foci  of  these  articles  also  generated  different  experimental  conditions. 
Because  Belanger  et  al.  are  attempting  to  replicate  wind  tunnel  experiments,  they  ‘release’  then- 
searchers  near  the  center  downwind  edge  of  a  1  by  2  m  search  area  with  the  odor  source  at  the 
center  of  the  upwind  edge.  Note  that  the  release  is  2  m  directly  downwind  from  the  odor  source. 
The  flow  is  essentially  laminar.  Therefore,  although  their  intermittency  characteristics  are 


tunable,  the  plume  has  no  meander.  The  best  source-finding  performance  presented  in  (Belanger 
and  Willis  1996a,  1996b,  1998;  Belanger  and  Arbas,  1998)  is  56%  success  for  a  300  s  duration 
simulation.  Herein,  we  have  studied  plume  tracking  over  a  100  by  100  m  area.  The  flow 
changes  with  space  and  time,  generating  significant  plume  meander  and  decorrelation  of  the  wind 
direction,  plume  centerline,  and  direction  to  the  source  as  the  distance  from  the  source  increases. 
Also,  searchers  are  released  on  a  10  by  10  grid  of  starting  locations  which  include  searchers 
starting  up  to  20  m  upstream  of  the  source  and  45  m  offset  from  the  mean  plume  location  (y=0) 
in  the  crosswind  direction.  The  success  rate  of  Strategy  in  in  finding  the  source  averaged  over  all 
starting  locations  was  97%  for  a  300  s  duration  simulation.  Therefore,  it  is  interesting  to 
consider  possible  reasons  for  this  improvement  in  performance  for  a  significantly  more  difficult 
problem.  A  first  reason  is  the  difference  in  objectives.  An  objective  of  our  simulations  was  to 
start  from  biological  models  and  to  improve  their  performance,  whereas  the  objective  of  Belanger 
et  al.  was  to  evaluate  strategies  relative  to  models  in  the  literature.  Optimization  of  their  models 
is  further  considered  in  (Belanger  and  Willis,  1998).  Second,  we  decompose  the  problem  into 
four  parts:  plume  finding,  plume  maintaining,  plume  reacquisition,  and  source  declaration. 
Casting  is  a  portion  of  the  plume  reacquisition  behavior.  If  plume  reacquisition  does  not  succeed 
within  a  few  seconds,  then  we  revert  to  plume  finding  behaviors.  In  contrast,  the  strategies  of 
Belanger  et  al.  continue  casting  until  they  detect  odor.  As  they  state,  many  of  their  simulations 
failed  because  the  moth  moved  so  far  from  the  plume  that  subsequent  casting  maneuvers  never 
again  brought  the  moth  into  contact  with  the  plume.  Third,  the  timing  of  the  Belanger 
Countertums  is  set  by  an  internal  clock  and  the  post-detection  behavior  may  command  a  course 
that  is  due  upwind.  Such  a  course  may  result  in  the  subsequent  countertum  taking  the  searcher 
well  outside  the  plume.  The  reacquisition  countertums  and  the  in-plume  maneuvers  of  Strategy 


HI  are  defined  to  increase  the  likelihood  that  the  first  turn  upon  leaving  the  plume  is  in  the  correct 
direction.  The  importance  of  the  sign  of  this  first  turn  is  also  stated  on  p.  358  of  (Belanger  and 
Arbas,  1998). 

7  CAVEATS  AND  CONCLUSIONS 

The  strategies  that  would  enable  robotic  vehicles  to  follow  a  meandering,  filamentous  odor 
plume  upwind  to  its  point  of  origin  must  be  robust  to  a  range  of  environmental  conditions. 
Optimization  of  an  overall  strategy  for  localization  of  an  upwind  and  distant  chemical  source  is  a 
difficult  problem.  Therefore,  the  overall  problem  was  decomposed  into  plume  finding,  plume 
maintaining,  plume  reacquisition,  and  source-found  declaration.  The  motivation  for 
decomposition  was  inspired  by  the  known  maneuvers  of  plume  tracing  by  biological  entities, 
particularly  flying  moths.  The  decomposition  has  been  fruitful  in  the  sense  that  each  portion  of 
the  decomposition  has  proved  tractable  and  successful  strategies  have  been  defined. 
Implementation  of  an  overall  strategy  requires  fusion  of  these  behaviors  into  an  overall  approach, 
with  the  appropriate  behavior  being  selected  at  each  time  instant.  The  literature  on  behavior 
based  robotics  is  useful  for  this  research  direction  (Brooks,  1986;  Li,  1994). 

The  main  objective  of  our  simulations  was  to  design  sensor,  signal  processing,  and 
actuation  algorithms  to  enable  autonomous  vehicles  to  track  chemical  plumes  to  their  source. 
Secondary  objectives  were  to  improve  the  understanding  of  the  nature  of  this  problem  and  the 
methods  that  biological  entities  use  for  its  solution.  Our  approach  has  involved  ‘optimization’  of 
strategy  parameters  and  redesign  of  algorithms  based  on  successive  iterations  of  the  optimization 
process.  Because  the  parameters  of  each  strategy  are  interrelated,  multiple  iterations  of  the 
optimization  could  further  improve  the  performance  (i.e.,  multiple  iterations  of  a  line  search 
approach);  therefore,  the  parameters  of  the  strategies  are  not  strictly  ‘optimal.’  We  did  not  pursue 


full  optimization  for  two  reasons.  First,  such  an  optimization  would  be  tuned  to  the  particular 
simulation  enviromnent  of  the  optimization  process.  An  additional  important  criterion  is 
robustness  of  performance  to  environmental  factors.  Second,  we  were  most  interested  in  the 
structural  features  of  the  strategies  that  were  required  to  yield  good  and  robust  performance.  A 
single  iteration  of  the  optimization  process  was  used  to  understand  the  tradeoffs  involved  in 
parameter  selection  and  to  select  reasonably  good  choices  for  the  strategy  parameters. 

The  main  structural  features  that  we  identified  related  to  strategy  design  are  as  follows. 
First,  maintaining  intermittent  contact  with  the  plume  is  critical  to  success.  Flow  direction  while 
detecting  the  odor  is  critical  information  indicative  of  direction  to  the  source.  If  contact  with  the 
plume  is  lost  for  more  than  a  few  seconds,  then  the  searcher  could  spend  large  amounts  of  time 
and  energy  searching  to  reacquire  contact  with  the  plume  without  making  progress  toward  the 
source.  Therefore,  the  strategy  design  approaches  described  in  this  article  selected  maintain  time 
while  making  progress  along  the  plume  as  a  key  criterion  to  be  minimized.  Second,  the  ability  of 
a  searcher  to  maintain  intermittent  contact  with  a  plume  is  determined  by  its  ability  to  turn  in  the 
correct  direction  (back  toward  the  plume)  if  it  leaves  the  plume.  Our  analysis  has  shown  that 
cross-plume  travel  with  countertuming  significantly  increases  the  likelihood  and  robustness  (to 
sensor  noise)  of  turning  in  the  correct  direction.  Becasue  the  countertum  is  performed  after 
leaving  the  plume  and  the  countertum  involves  turning  through  the  upwind  direction,  this  results 
in  the  searcher  making  some  (possibly  most)  of  its  upwind  progress  while  it  is  out  of  the  plume. 
Similar  up-plume  progress  while  out  of  the  plume  has  been  observed  in  the  field  with  released 
gypsy  moths  (David  et  al.,  1983).  In  those  observations  of  moth  flight  within  15  m  of  a  point 
source  of  pheromone  in  a  open  field,  the  instantaneous  wind  direction  usually  pointed  toward  the 
odor  source  (cf.  Elkinton  et  al,  1994;  Brady  et  al.,  1987). 
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The  overall  objective  is  to  maneuver  the  vehicle  to  the  odor  source  location  in  a  resource 
(e.g.,  time,  energy)  efficient  manner.  Different  strategies  will  result  from  different  weightings  of 
the  resources.  For  example,  time  optimality  (e.g.,  an  advantage  in  rapid  mate  finding)  will  result 
in  higher  energy  usage  than  energy  optimality.  Also,  the  penalty  or  reward  functions  that  are 
defined  have  a  strong  effect  on  the  definition  of  the  optimal  strategy.  For  the  autonomous  vehicle 
plume  tracing  application,  the  total  time  and  energy  expended  in  the  search  do  not  need  to  be 
minimized,  but  they  do  need  to  satisfy  the  mission  time  and  vehicle  energy  capacity  constraints. 
Also,  the  mission  success  is  mainly  evaluated  by  whether  the  vehicle  ultimately  and  accurately 
determines  the  odor  source  location.  Therefore,  the  vehicle  plume  tracing  application  is  closer  to 
an  organism  finding  a  long-lasting  resource,  such  as  a  plant,  rather  than  being  the  first  to  locate  a 
prospective  mate.  Strategy  ID  developed  herein  is  remarkably  successful  in  finding  an  odor 
source  (i.e.,  97%)  with  a  high  accuracy  (i.e.,  within  1  m)  within  300  s  for  a  100  m  square  search 
area.  This  is  well  within  reasonable  vehicle  energy  and  mission  time  constraints.  This 
performance  was  also  robust  across  simulated  air  plumes  and  experimental  water  flume  data. 
Verification  of  this  performance  awaits  testing  of  these  plume-following  strategies  with  robots  in 
natural  environments. 

Section  3  includes  a  heuristic  definition  of  ‘in  the  plume.’  Various  alternative  definitions 
could  be  suggested.  A  main  criterion  for  this  definition  is  that  the  searcher  itself  must  be  able  to 
evaluate  this  definition  based  on  its  recent  sensory  information.  Therefore,  time-average 
contours  or  out-of-plane  images  are  not  relevant.  Also,  a  decision  based  on  the  instantaneous 
sensed  concentration  is  overly  sensitive,  because  turbulent  diffusion  is  known  to  result  in  patches 
of  above  and  below  threshold  chemical  within  the  plume  body  with  the  pulse  duration  and  pulse 
return  distributions  determined  by  the  flow  conditions.  The  definition  of  Section  3  was  selected 


since  it  addressed  the  above  issues  and  was  efficiently  computable  by  both  engineered  and 
biological  entities. 

We  also  have  assumed  a  planar  world  in  which  the  plume’s  (unspecified)  height  above 
ground  is  essentially  unvarying.  In  nature,  plumes  undulate  to  some  extent  and,  under  some  non- 
adiabatic  (unstable)  atmospheric  conditions,  may  rise  vertically  (e.g.,  Fares,  Sharpe  and 
Maguson,  1980;  Schal,  1986)  or  fall.  Certainly  flying  insects  must  navigate  plumes  in  3-D, 
whereas  organisms  such  as  lobsters  and  blue  crabs  sample  the  odor  relatively  close  to  the 
substrate  on  which  they  walk.  Moths  stabilize  their  flight  altitude  via  an  optomotor  reaction, 
most  likely  with  feedback  from  the  equatorial  region  of  the  eyes  (Vickers  and  Baker,  1994b).  An 
example  of  the  importance  of  the  two  dimensional  application,  as  described  in  Stacey  (2000), 
occurs  when  stratification  of  the  fluid  flow  can  result  in  an  effectively  2-D  search  problem  even 
in  a  3-D  environment. 

Many  features  of  the  output  of  our  three  navigation  strategies  superficially  resemble  the 
general  zigzag  tracks  of  moths  and  other  organisms  orienting  upwind  along  an  odor  plume.  Even 
though  we  have  explicitly  designed  these  strategies  to  be  biomimetic,  in  the  sense  that  they  use 
many  described  features  of  sensory  perception  and  a  navigational  strategy  of  countertuming 
across  the  windline  while  heading  upwind,  we  cannot  be  certain  that  we  have  employed  identical 
causal  mechanisms  (Webb,  2000).  This  difficulty  is  illustrated  by  the  ‘robolobster,’  a  lobster 
biomimic  which  has  been  designed  to  track  an  odor  plume  in  a  water  flow  using  artificial  odor 
sensors  that  are  positioned  to  match  a  lobster’s  chemosensory  array.  These  robots  successfully 
track  an  odor-scented  plume  upstream  over  several  meters,  but  the  precise  form  of  the  tracks  and 
success  rates  of  the  robolobster  and  lobsters  differ  (Grasso,  Consi,  Mountain  and  Atema,  2000). 
Such  a  divergence  in  behavioral  outputs  indicates  that  the  robot  and  the  lobsters  may  not  be 


extracting  information  about  odor  flux  in  precisely  the  same  way,  that  the  real  time  control  of 
movement  by  the  lobster  has  not  been  replicated  in  the  robot,  or  that  other  sensory  cues,  such  as 
the  direction  of  fluid  movement,  need  to  be  incorporated  into  the  animat’s  response. 
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APPENDIX 

The  main  body  of  this  paper  focuses  on  the  problem  of  designing  a  strategy  to  maintain 
contact  with  an  intermittent  plume.  Some  of  the  evaluations  in  the  paper  also  require  a  strategy 
for  the  searcher  to  find  the  plume  and  a  method  to  declare  the  successful  completion  of  a  search. 
Both  of  these  issues  are  discussed  in  this  appendix. 

Plume  Finding 

The  article  considers  plume  tracing  within  a  rectangular  search  area  with  comers  defined 
by  (&y y),(x,  y),  and  (x,  y).  The  initial  searcher  location  (xq,  yg)  is  assumed  to  be  located 

in  the  search  area.  The  plume  finding  behavior  is  defined  by  crosswind  search  within  the 
rectangular  search  area  and  reflection  off  the  boundaries  of  the  search  area  when  encountered. 
Initially,  the  vehicle  heading  is  defined  as 

(//ft)  =  y/w(t)  +  d0  sign(Q;  +  y  )/2-yg) 

with  do  initialized  to  70°  where  y/w(t)  is  the  wind  direction.  This  results  in  initial  crosswind  flight 
towards  the  furthest  y  boundary.  This  heading  is  maintained  until  either  odor  is  detected,  in  which 
case  the  strategy  switches  to  the  maintain  behavior,  or  the  edge  of  the  search  region  is 
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encountered.  If  a  y  boundary  is  encountered  (i.e.,  *  e  [x,  x]  and  y  g  [y,  y]),  then  the  vehicle 
heading  is  defined  by 

y/v(t)  =  y/w(t)  +  0,+i  (i.e.,  reflection  reverses  crosswind  direction) 

where  0,+j  =  -6,  and  i  is  incremented  by  1  at  the  time  Ty  at  which  the  y  boundary  was  encounter. 

If  an  x  boundary  is  encountered  (i.e.,  x  g  [x,  x ]  and  y  e  [y,  y ]),  then  the  vehicle  heading  is 

defined  by 

Vv(t)  ~  y/w(t)  +  0j+i 

where 

0i+i  =  70  sign (15, )  if  x  <  x  (i.e.,  reflection  to  slightly  upwind  search) 

0i+i  =  110  sign (6i )  if  x  >  x  (i.e.,  reflection  to  slightly  downwind  search) 

and  i  is  incremented  by  1  at  the  time  Tx  at  which  the  x  boundary  was  encounter.  This  plume 

finding  behavior  continues  until  the  plume  is  found  or  the  maximum  simulation  time  is 
exceeded. 

Source  Finding  Declaration 

The  evaluations  herein  did  not  consider  the  problem  of  the  searcher  declaring  that  the 
source  had  been  found.  Instead,  each  search  was  declared  to  have  successfully  found  the  source 
when  the  distance  from  searcher  to  source: 

P  ~  —  X  src  )  ^  O'  ~  y  src ) 

was  smaller  than  R=l.  When  this  occurs,  the  vehicle  is  considered  to  have  reached  the  odor 
source  and  the  simulation  is  stopped. 

Because  the  searcher  could  wander  randomly  and  meet  this  stopping  criteria,  the 
performance  of  such  a  random  search  was  evaluated  for  comparison.  The  results  of  the  random 
search  are  not  included,  because  the  random  search  takes  so  long  that  it  is  completely  ineffective 


(i.e.,  except  when  the  searcher  starts  very  near  the  source,  the  simulation  typically  timed  out 
unsuccessfully.). 


Vv(t)  =  y/u(t) 

Vv(t)  =  iffu(t)  -y 
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Strategy 

Maintain  Behavior 

Reacquisition  Behavior 

Parameters 

I 

If  the  searcher  has  detected 

If  the  searcher  has  detected  odor  within  the 

X=  1 

Upwind 

odor  within  the  last 

last  Tw  seconds,  but  not  the  last  X  seconds, 

Tw  “4,  and 

with  wind 

relative 

countertums 

X  seconds,  then  the  searcher 

travels  straight  upwind. 

then  the  searcher  travels  at  an  angle  y= 

±90°  to  upwind.  The  sign  of  yis  defined 

based  on  the  vehicle  heading  and  wind 

direction  to  cause  the  vehicle  to  reverse  its 

crosswind  direction  (i.e.,  countertum). 

y  =  ±90° 

n 

If  the  searcher  is  currently 

If  the  searcher  has  detected  odor  within  the 

A=l, 

Constant 

sensing  odor,  it  turns  toward 

last  7>  seconds,  but  not  the  last  X  seconds, 

Tw~ 4, 

crosswind 

straight  upwind.  If  the 

then  the  searcher  travels  at  an  angle  y= 

(3=  35,  and  ! 

with 

countertum 

searcher  is  not  currently 

sensing  odor,  but  has  sensed 

odor  within  the  last 

X  seconds,  then  the  searcher 

travels  at  an  angle  [J  with 

respect  to  upwind. 

±90°  to  upwind.  The  sign  of  yis  defined  as 

the  opposite  of  the  sign  of  j3.  This 

generates  countertuming  travel  and  the  sign 

of  yis  not  sensitive  to  the  measured  wind 

direction  or  vehicle  heading. 

y  =  ±90° 

m 

If  it  has  detected  odor 

If  the  searcher  has  detected  odor  within  the 

X=l, 

Progressive 

within  the  last  X  seconds, 

last  Tw  seconds,  but  not  the  last  X  seconds, 

Tw— 4, 

crosswind 

then  the  searcher  travels  at 

then  the  searcher  travels  at  an  angle  y= 

m 

with 

an  angle  /?(/)  relative  to 

±90°  to  upwind.  The  sign  of  yis  defined  as 

described 

countertum 

upwind,  where  t  is  the  time 

since  odor  was  first  detected 

on  the  present  crosswind 

traverse  of  the  plume. 

the  opposite  of  the  sign  of  /?.  This 

generates  countertuming  travel  and  the  sign 

of  yis  not  sensitive  to  the  measured  wind 

direction  or  vehicle  heading. 

by  (12), 

and 

y  =  ±90° 

Table  3.  Summary  of  the  Strategies  Specified  in  Sections  3  and  4. 
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IT 
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[x,  xj 
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¥r(t) 

¥v(t) 


Units 
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Deg 

Deg 

Deg 


Deg 

Deg 
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m 


Deg 

Deg 

Deg 


Interpretation 

Sensor  detection  threshold 

Coordinates  of  searcher 

Time  from  last  above  threshold  detection  until  the  searcher  begins  the  plume 
reacquisition  behavior 

Time  from  last  above  threshold  detection  the  searcher  reverts  to  the  plume 
finding  behavior 

Time  to  first  detect  the  odor,  depends  on  starting  location  and  time. 

Time  in  contact  with  the  plume 

Total  time  to  recontact  the  plume  after  its  contact  is  lost 
Sensed  concentration  at  time  t 

Instantaneous  direction  of  the  wind  plus  180  deg  (i.e.,  upflow  direction) 

Angle  of  travel  relative  to  y/u(t)  during  the  maintain  plume  behavior 

Angle  of  travel  relative  to  if/u(t)  during  plume  reacquisition  reacquisition 

behavior 

Random  variable 

Standard  deviation 

Minimum  and  maximum  x  position 

Minimum  and  maximum  y  position 

¥w(t)  -  ¥v(t) 

Commanded  vehicle  heading  at  time  t 
Wind  heading  at  time  t  at  the  vehicle  location 


Table  Captions 


Table  1 .  Summary  of  Strategy  I. 


Table  2.  Summary  of  Strategy  II. 


Table  3.  Summary  of  the  Strategies  Specified  in  Sections  3  and  4. 


Figure  Captions 


Fig.l.  Concentration  time  series  with  threshold  t=2000. 

Fig.  2.  Vehicle  (not  to  scale)  and  plume  within  the  simulated  search  area.  The  x  coordinate  is 
between  x  =0  and  x=100  m,  and  is  positive  in  the  mean  downwind  (i.e.,  horizontal)  direction. 
The  y  coordinate  is  between  y  =  -50  and  y=  50  m,  and  is  positive  down.  Heading  is  measured 
positively  with  respect  to  the  z-axis  in  a  right-handed  coordinate  system.  The  grey  scale  indicates 
above  threshold  concentration.  The  arrows  indicate  the  magnitude  and  direction  of  the  local 
wind  vector  at  the  tail  of  the  arrow. 

Fig.  3.  Illustration  of  the  designed  searcher  behavior  ‘within  the  plume.’  Black  dots  depict  above 
threshold  patches  of  odor.  After  first  odor  detection,  the  searcher  travels  at  an  angle  p  relative  to 
upwind.  The  angle  p  is  maintained  until  above  threshold  odor  has  not  been  detected  for  X 
seconds,  at  which  time  the  searcher  countertums  to  travel  at  an  angle  y  with  respect  to  the  wind. 

Fig.  4.  Sensitivity  of  Tm  to  waiting  time  X  and  threshold  x. 

Fig.  5.  Mean  reacquisition  time  versus  the  angle  y  The  parameter  a  indicates  the  standard 
deviation  of  the  normal  random  variable  <j>.  (a)  Top  -  Mean  of  all  records  for  which  y<  0  (right 
of  upwind)  yielded  better  performance,  (b)  Bottom  -  Mean  of  all  records  for  which  y>  0  (left  of 
upwind)  yielded  better  performance. 

Fig.  6.  Contour  plot  of  the  average  time  to  find  the  odor  source  as  a  function  of  the  initial 
searcher  position  for  a  searcher  using  Strategy  1.  The  average  finding  time  for  each  initial 
position  is  evaluated  over  100  runs  from  the  starting  position.  The  large  meander  set  of  plume 
parameters  was  used.  The  nominal  speed  at  which  the  searcher  travels  is  1  m/s. 
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Fig.  7.  Mean  of  records  from  each  reacquisition  iteration  after  classification,  (a)  Left  -  Mean  of 
all  records  classified  as  category  1  (i.e.,  /?<  0  is  better),  (b)  Right  -  Mean  of  all  records  classified 
as  category  2  (i.e.,  /?  >  0  is  better). 

Fig.  8.  Depiction  of  the  plume  maintaining  behavior  with  a  time  varying  offset  relative  to  upwind 

P(t). 

Fig.  9.  Maintain  time  dependence  on  strategy  and  environment  parameters.  Each  plotted  point  is 
the  average  over  100  repetitions.  Note  the  different  scales.  Left  -  Maintain  time  versus  the  angle 
p.  The  top  row  of  graphs  use  simulation  parameters  resulting  in  large  amplitude,  low  frequency 
plume  meander.  The  bottom  row  of  graphs  use  simulation  parameters  resulting  in  small 
amplitude,  high  frequency,  plume  meander.  The  bottom  curve  on  each  of  the  left  column  graphs 
uses  a  constant  /?.  The  upper  curve  on  each  of  the  left  column  graphs  uses  the  time  varying  P(t). 
For  the  P(t)  curves,  the  horizontal  axis  should  be  read  as  the  maximum  of  P(t)  (i.e.  3pi  in  the 
definition  of  p(t)  after  eqn  (11)).  Right  top—  Maintain  time  versus  vehicle  speed  during 
reacquisition  phase.  Right  bottom-  Maintain  time  versus  vehicle  speed  during  maintain  phase. 

Fig.  10.  Time  to  find  the  source  versus  initial  x-axis  starting  position  for  the  three  strategies 
summarized  in  Table  3.  The  starting  positions  were  (i,0)  for  i=0,  ...,  100.  Performance  from  each 
starting  location  was  evaluated  500  times. 

Fig.  11.  Contour  plots  of  Strategy  IE  performance  as  measured  by  the  total  time  to  maneuver  to 
within  1  m  of  the  source  location.  The  four  plots  correspond  to  results  using  the  plume 
simulation  and  three  experimental  flume  data  sets. 
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FOOTNOTE  TEXT 


1  Theoretically,  an  instantaneous  definition  of  the  plume  extent  could  be  defined  as  an  area  with  a 
high  probability  of  detecting  an  above  threshold  concentration  of  odor.  This  area  is  time  varying. 
Derivation  of  this  area  is  extremely  complex  and  would  require  the  search  vehicle  to  have 
knowledge  of  the  odor  source  location  and  the  fluid  flow  field  over  the  entire  search  region  for  a 
significant  range  of  past  time.  Therefore,  this  theoretically  appealing  definition  of  the  plume  is 
impractical  for  on- vehicle  maneuvering. 

2  A  single  simulation  image  is  contained  in  this  article.  The  time  varying  simulation  output  is 
more  interesting.  Therefore,  an  executable  version  (for  Windows)  of  the  simulation  described 
herein  is  available  for  download  at  http://www.ee.ucr.edu/~farrell.  Source  code  is  available  by 
contacting  the  second  author. 

3  The  vehicle  heading  dynamics  are  defined  by  —ZQ.  =  -ay/(t)  +  at//v  ( t )  for  a  >  0.  In  this  article,  a 

dt 

=  5.0.  Therefore,  ^converges  toward  y/v  as  t  increases.  Throughout  this  article,  y/v  should  be 
interpreted  as  a  commanded  reference  heading.  For  a  constant  commanded  heading, 
convergence  is  approximately  complete  after  (3 la)  seconds. 

4  Zigzagging  is  countertuming  with  significant  upwind  or  downwind  displacement  (Kennedy, 
1986). 

5  Note  that  when  the  searcher  leaves  the  plume  its  instantaneous  heading  will  typically  have  a 
significant  crosswind  component.  The  reacquisition  course  command  will  be  due  crosswind  in 
the  opposite  direction.  If  the  searcher  reverses  its  direction  with  a  turn  through  upwind,  then  the 
searcher  will  have  made  upwind  progress  while  out  of  the  plume.  Depending  on  the  parameter 


settings  and  the  plume  characteristics,  it  is  possible  that  the  searcher  may  make  the  majority  of  its 
upwind  progress  while  it  is  outside  the  plume.  In  such  a  situation,  the  searcher  would  only  be 
crossing  the  plume  between  upwind  ‘sprints.’  The  plume  crossings  may  ensure  that  the  upwind 
sprints  are  centered  on  the  plume  and  have  not  passed  the  odor  source.  The  issue  of  moths 
making  significant  progress  toward  the  odor  source  while  outside  the  plume  has  been  discussed 
in  the  moth  literature  (David,  Kennedy  and  Ludlow,  1983;  Ludlow,  1983). 


